Dr. Kevin B. Johnson delivered this address on May 16, 2026, at the Commencement Ceremony of The D. Bradley McWilliams School of Biomedical Informatics, UTHealth Houston, to the graduating class of 2026. The address uses the concept of "The Big Mo" (compounding momentum) as a frame for understanding the current inflection point in AI and medicine. Drawing on his own career arc from paper-based clinical practice at Johns Hopkins through early adoption of health informatics to the present era of AI in healthcare, Johnson argues that the fears graduates hold about technological obsolescence and institutional instability are real but misdirected. He reframes both: biomedical informatics professionals are not targets of AI but its essential architects, and the external environment has always been uncertain for those doing important work. His charge to graduates is singular: stay on the wave.

OBJECTIVE: To support ambulatory care innovation, we created Observer, a multimodal dataset comprising videotaped outpatient visits, electronic health record (EHR) data, and structured surveys. This paper describes the data collection procedures and summarizes the clinical and contextual features of the dataset. MATERIALS AND METHODS: A multistakeholder steering group shaped recruitment strategies, survey design, and privacy-preserving design. Consented patients and primary care providers (PCPs) were recorded using room-view and egocentric cameras. EHR data, metadata, and audit logs were also captured. A custom de-identification pipeline, combining transcript redaction, voice masking, and facial blurring, ensured video and EHR HIPAA compliance. RESULTS: We report on the first 100 visits in this continually growing dataset. Thirteen PCPs from 4 clinics participated. Recording the first 100 visits required approaching 210 patients, from which 129 consented (61%), with 29 patients missing their scheduled encounter after consenting. Visit lengths ranged from 5 to 100 minutes, covering preventive care to chronic disease management. Survey responses revealed high satisfaction: 4.24/5 (patients) and 3.94/5 (PCPs). Visit experience was unaffected by the presence of video recording technology. DISCUSSION: We demonstrate the feasibility of capturing rich, real-world primary care interactions using scalable, privacy-sensitive methods. Room layout and camera placement were key influences on recorded communication and are now added to the dataset. The Observer dataset enables future clinical AI research/development, communication studies, and informatics education among public and private user groups. CONCLUSION: Observer is a new, shareable, real-world clinic encounter research and teaching resource with a representative sample of adult primary care data.

OBJECTIVES: We report on findings from a meeting convened by the American College of Medical Informatics (ACMI) to characterize aspects of the patient experience that could be improved using informatics. MATERIALS AND METHODS: The American College of Medical Informatics fellows were invited to share their experiences as patients and suggest informatics approaches that may improve the patient experience. RESULTS: We identified 4 themes: (1) getting the right care, (2) data sharing and data interoperability, (3) guiding low-cost evaluations, and (4) predictive analytics. DISCUSSION: Despite widespread adoption of health IT, patient experiences remain far from optimal. CONCLUSION: The American College of Medical Informatics fellows identified informatics approaches, applications, and research areas that have the potential to improve patient experiences with health care systems.

Multimodal large language models (MLLMs) can simultaneously process visual, textual, and auditory data, capturing insights that complement human analysis. However, existing video question-answering (VidQA) benchmarks and datasets often exhibit a bias toward a single modality, despite the goal of requiring advanced reasoning skills that integrate diverse modalities to answer the queries. In this work, we introduce the modality importance score (MIS) to identify such bias. It is designed to assess which modality embeds the necessary information to answer the question. Additionally, we propose an innovative method using state-of-the-art MLLMs to estimate the modality importance, which can serve as a proxy for human judgments of modality perception. With this MIS, we demonstrate the presence of unimodal bias and the scarcity of genuinely multimodal questions in existing datasets. We further validate the modality importance score with multiple ablation studies to evaluate the performance of MLLMs on permuted feature sets. Our results indicate that current models do not effectively integrate information due to modality imbalance in existing datasets. Our proposed MLLM-derived MIS can guide the curation of modality-balanced datasets that advance multimodal learning and enhance MLLMs' capabilities to understand and utilize synergistic relations across modalities.

OBJECTIVE: The American Medical Informatics Association (AMIA) Task Force on Diversity, Equity, and Inclusion (DEI) was established to address systemic racism and health disparities in biomedical and health informatics, aligning with AMIA's mission to transform healthcare. AMIA's DEI initiatives were spurred by member voices responding to police brutality and COVID-19's impact on Black/African American communities. MATERIALS AND METHODS: The Task Force, consisting of 20 members across 3 groups aligned with AMIA's 2020-2025 Strategic Plan, met biweekly to develop DEI recommendations with the help of 16 additional volunteers. These recommendations were reviewed, prioritized, and presented to the AMIA Board of Directors for approval. RESULTS: In 9 months, the Task Force (1) created a logic model to support workforce diversity and raise AMIA's DEI awareness, (2) conducted an environmental scan of other associations' DEI activities, (3) developed a DEI framework for AMIA meetings, (4) gathered member feedback, (5) cultivated DEI educational resources, (6) created a Board nominations and diversity session, (7) reviewed the Board's Strategic Planning for DEI alignment, (8) led a program to increase diversity at the 2020 AMIA Virtual Annual Symposium, and (9) standardized socially-assigned race and ethnicity data collection. DISCUSSION: The Task Force proposed actionable recommendations that focused on AMIA's role in addressing systemic racism and health equity, helping the organization understand its member diversity. CONCLUSION: This work supported marginalized groups, broadened the research agenda, and positioned AMIA as a DEI leader while reinforcing the need for ongoing transformation within informatics.

INTRODUCTION: The rapid advancement of artificial intelligence (AI) has led to significant transformations in health and healthcare. As AI technologies continue to evolve, there is an urgent need to establish a unified framework that guides the design, implementation, and evaluation of AI-driven interventions across individual and population health contexts. APPROACH: In response to this need, the National Academy of Medicine (NAM) has initiated the development of an AI code of conduct (AICC) through its Digital Health Action Collaborative. This code of conduct is grounded in shared principles and commitments, aiming to actualize ethical and effective AI practices within the broader health and healthcare ecosystem. Given its specialized expertise and insight, the biomedical informatics (BMI) community plays a pivotal role in shaping and applying these guidelines. RECOMMENDATIONS: We, as members of the AICC Steering Committee and the NAM Digital Health Action Collaborative, urge BMI educators, researchers, and practitioners to engage actively in refining and implementing the AICC. This involvement is critical to ensuring that the code is robust, applicable, and continuously improved to meet the evolving challenges facing health and healthcare.

INTRODUCTION: Automation of clinical orders in electronic health records (EHRs) has the potential to reduce clinician burden and enhance patient safety. However, determining which orders are appropriate for automation requires a structured framework to ensure clinical validity, transparency, and safety. OBJECTIVE: To develop and validate a framework of desiderata for assessing the appropriateness of automating clinical orders in EHRs and to demonstrate its operational value in a live health system dataset. MATERIALS AND METHODS: The study comprised 4 phases to move from concept generation to real-world demonstration. First, we conducted focus group analyses using ground theory to identify themes and developed desiderata informed by these themes and existing literature. We validated the desiderata by surveying clinicians at a single institution, presenting 10 use cases to and assessing perceived appropriateness, cognitive support, and patient safety using a 4-point Likert scale. Survey results were compared to a priori appropriateness designations using t-tests. To evaluate operational impact, we analyzed one year of order-based alerts and orders (1.4 million firings alert and 44.1 million orders, respectively) using filtering rules and association rule mining to identify candidate orders for automation and their impact. RESULTS: We identified 8 desiderata for automated order appropriateness: logical consistency, data provenance, order transparency, context permanence, monitoring plans, trigger consistency, care team empowerment, and system accountability. Use cases deemed appropriate based on these criteria received significantly higher scores for appropriateness (3.13 ± 0.84 vs 2.30 ± 0.99), cognitive support (3.08 ± 0.82 vs 2.25 ± 0.94), and patient safety (3.08 ± 0.86 vs 2.21 ± 0.98) (all P < .001) compared to those considered inappropriate. Operational analysis revealed an alert firing 19 109 times annually, with a 96% signed order rate, where automation could save an estimated 26.5 provider hours per year. Additionally, an association rule with 16 628 occurrences (68.4% confidence) suggested automation could save 15.8 hours annually and yield 8000 additional appropriate orders. DISCUSSION: The desiderata align with clinician perceptions and provide a structured approach for evaluating automated orders. Our findings highlight the potential for automation of certain clinical orders to improve cognitive support while maintaining patient safety. CONCLUSION: Healthcare systems should use these desiderata, coupled with data mining techniques, to systematically identify and govern appropriate automated orders. Further research is needed to validate operational scalability.

OBJECTIVE: The increasing use of audio-video (AV) data in healthcare has improved patient care, clinical training, and medical and ethnographic research. However, it has also introduced major challenges in preserving patient-provider privacy due to Protected Health Information (PHI) in such data. Traditional de-identification methods are inadequate for AV data, which can reveal identifiable information such as faces, voices, and environmental details. Our goal was to create a pipeline for de-identifying AV healthcare data that minimized the human effort required to guarantee successful de-identification. METHODS: We combined open-source tools with novel methods and infrastructure into a six-stage pipeline: (1) transcript extraction using WhisperX, (2) transcript de-identification with an adapted PHIlter, (3) audio de-identification through scrubbing, (4) video de-identification using YOLOv11 for pose detection and blurring, (5) recombining de-identified audio and video, and (6) validation and correction via manual quality control (QC). We developed two de-identification strategies to support different tolerances for lossy video images. We evaluated this pipeline using 10 h of simulated clinical AV recordings, comprising nearly 1.1 million video frames and approximately 72,000 words. RESULTS: In Precision Privacy Preservation (PPP) mode, MedVidDeId achieved a success rate of 50%, while in Greedy Privacy Preservation (GPP) mode, it achieved a 97.5% success rate. Compared to manual methods for a 15 min video segment, the pipeline reduced de-identification time by 26.7% in PPP and 64.2% in GPP modes. CONCLUSION: The MedVidDeID pipeline offers a viable, efficient hybrid solution for handling AV healthcare data and privacy preservation. Future work will focus on reducing upstream errors at each stage and minimizing the role of the human in the loop.

OBJECTIVES: Automated phenotyping algorithms can reduce development time and operator dependence compared to manually developed algorithms. One such approach, PheNorm, has performed well for identifying chronic health conditions, but its performance for acute conditions is largely unknown. Herein, we implement and evaluate PheNorm applied to symptomatic COVID-19 disease to investigate its potential feasibility for rapid phenotyping of acute health conditions. MATERIALS AND METHODS: PheNorm is a general-purpose automated approach to creating computable phenotype algorithms based on natural language processing, machine learning, and (low cost) silver-standard training labels. We applied PheNorm to cohorts of potential COVID-19 patients from 2 institutions and used gold-standard manual chart review data to investigate the impact on performance of alternative feature engineering options and implementing externally trained models without local retraining. RESULTS: Models at each institution achieved AUC, sensitivity, and positive predictive value of 0.853, 0.879, 0.851 and 0.804, 0.976, and 0.885, respectively, at quantiles of model-predicted risk that maximize F1. We report performance metrics for all combinations of silver labels, feature engineering options, and models trained internally versus externally. DISCUSSION: Phenotyping algorithms developed using PheNorm performed well at both institutions. Performance varied with different silver-standard labels and feature engineering options. Models developed locally at one site also worked well when implemented externally at the other site. CONCLUSION: PheNorm models successfully identified an acute health condition, symptomatic COVID-19. The simplicity of the PheNorm approach allows it to be applied at multiple study sites with substantially reduced overhead compared to traditional approaches.

OBJECTIVE: Large-language models (LLMs) can potentially revolutionize health care delivery and research, but risk propagating existing biases or introducing new ones. In epilepsy, social determinants of health are associated with disparities in care access, but their impact on seizure outcomes among those with access remains unclear. Here we (1) evaluated our validated, epilepsy-specific LLM for intrinsic bias, and (2) used LLM-extracted seizure outcomes to determine if different demographic groups have different seizure outcomes. MATERIALS AND METHODS: We tested our LLM for differences and equivalences in prediction accuracy and confidence across demographic groups defined by race, ethnicity, sex, income, and health insurance, using manually annotated notes. Next, we used LLM-classified seizure freedom at each office visit to test for demographic outcome disparities, using univariable and multivariable analyses. RESULTS: We analyzed 84 675 clinic visits from 25 612 unique patients seen at our epilepsy center. We found little evidence of bias in the prediction accuracy or confidence of outcome classifications across demographic groups. Multivariable analysis indicated worse seizure outcomes for female patients (OR 1.33, P ≤ .001), those with public insurance (OR 1.53, P ≤ .001), and those from lower-income zip codes (OR ≥1.22, P ≤ .007). Black patients had worse outcomes than White patients in univariable but not multivariable analysis (OR 1.03, P = .66). CONCLUSION: We found little evidence that our LLM was intrinsically biased against any demographic group. Seizure freedom extracted by LLM revealed disparities in seizure outcomes across several demographic groups. These findings quantify the critical need to reduce disparities in the care of people with epilepsy.

OBJECTIVE: We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts. MATERIALS AND METHODS: We developed a 10-step Clickbusting process and implemented a program that included a curriculum, CDS alert inventory, oversight process, and gamification. We carried out two 3-month rounds of the Clickbusters program at VUMC. We completed descriptive analyses of the changes made to alerts during the process, and of alert firing rates before and after the program. RESULTS: Prior to Clickbusters, VUMC had 419 CDS alerts in production, with 488 425 firings (42 982 interruptive) each week. After 2 rounds, the Clickbusters program resulted in detailed, comprehensive reviews of 84 CDS alerts and reduced the number of weekly alert firings by more than 70 000 (15.43%). In addition to the direct improvements in CDS, the initiative also increased user engagement and involvement in CDS. CONCLUSIONS: At VUMC, the Clickbusters program was successful in optimizing CDS alerts by reducing alert firings and resulting clicks. The program also involved more users in the process of evaluating and improving CDS and helped build a culture of continuous evaluation and improvement of clinical content in the electronic health record.

Clinical informatics remains underappreciated among medical students in part due to a lack of integration into undergraduate medical education (UME). New developments in the study and practice of medicine are traditionally introduced via formal integration into undergraduate medical curricula. While this path has certain advantages, curricular changes are slow and may fail to showcase the breadth of clinical informatics activities. Less formal and more flexible approaches can circumvent these drawbacks. Interest groups (IGs), which are organized through the Association of American Medical College Careers in Medicine (CiM) program, exemplify the informal approach. CiM IGs are student-led groups that provide exposure to different specialty options, acting as an adjunct to the traditional medical curriculum. While the primary purpose of these groups is to assist students applying to residency programs, we took a novel approach of using an IG to increase student exposure to an area of medicine that had not yet been formally integrated at our institution. IGs provide unique advantages to formal integration into a curriculum as they can be more easily setup and can quickly respond to student interests. Furthermore, IGs can act synergistically with UME, acting as proving grounds for ideas that can lead to new courses. We believe that the lessons and takeaways from our experience can act as a guide for those interested in starting similar organizations at their own schools.

OBJECTIVE: The Genomic Medicine Working Group of the National Advisory Council for Human Genome Research virtually hosted its 13th genomic medicine meeting titled "Developing a Clinical Genomic Informatics Research Agenda". The meeting's goal was to articulate a research strategy to develop Genomics-based Clinical Informatics Tools and Resources (GCIT) to improve the detection, treatment, and reporting of genetic disorders in clinical settings. MATERIALS AND METHODS: Experts from government agencies, the private sector, and academia in genomic medicine and clinical informatics were invited to address the meeting's goals. Invitees were also asked to complete a survey to assess important considerations needed to develop a genomic-based clinical informatics research strategy. RESULTS: Outcomes from the meeting included identifying short-term research needs, such as designing and implementing standards-based interfaces between laboratory information systems and electronic health records, as well as long-term projects, such as identifying and addressing barriers related to the establishment and implementation of genomic data exchange systems that, in turn, the research community could help address. DISCUSSION: Discussions centered on identifying gaps and barriers that impede the use of GCIT in genomic medicine. Emergent themes from the meeting included developing an implementation science framework, defining a value proposition for all stakeholders, fostering engagement with patients and partners to develop applications under patient control, promoting the use of relevant clinical workflows in research, and lowering related barriers to regulatory processes. Another key theme was recognizing pervasive biases in data and information systems, algorithms, access, value, and knowledge repositories and identifying ways to resolve them.

Open discussions of social justice and health inequities may be an uncommon focus within information technology science, business, and health care delivery partnerships. However, the COVID-19 pandemic-which disproportionately affected Black, indigenous, and people of color-has reinforced the need to examine and define roles that technology partners should play to lead anti-racism efforts through our work. In our perspective piece, we describe the imperative to prioritize TechQuity-equity and social justice as a technology business strategy-through collaborating in partnerships that focus on eliminating racial and social inequities.

OBJECTIVE: The study sought to provide physicians, informaticians, and institutional policymakers with an introductory tutorial about the history of medical documentation, sources of clinician burnout, and opportunities to improve electronic health records (EHRs). We now have unprecedented opportunities in health care, with the promise of new cures, improved equity, greater sensitivity to social and behavioral determinants of health, and data-driven precision medicine all on the horizon. EHRs have succeeded in making many aspects of care safer and more reliable. Unfortunately, current limitations in EHR usability and problems with clinician burnout distract from these successes. A complex interplay of technology, policy, and healthcare delivery has contributed to our current frustrations with EHRs. Fortunately, there are opportunities to improve the EHR and health system. A stronger emphasis on improving the clinician's experience through close collaboration by informaticians, clinicians, and vendors can combine with specific policy changes to address the causes of burnout. TARGET AUDIENCE: This tutorial is intended for clinicians, informaticians, policymakers, and regulators, who are essential participants in discussions focused on improving clinician burnout. Learners in biomedicine, regardless of clinical discipline, also may benefit from this primer and review. SCOPE: We include (1) an overview of medical documentation from a historical perspective; (2) a summary of the forces converging over the past 20 years to develop and disseminate the modern EHR; and (3) future opportunities to improve EHR structure, function, user base, and time required to collect and extract information.

Our goal is to summarize the collective experience of 15 organizations in dealing with uncoordinated efforts that result in unnecessary delays in understanding, predicting, preparing for, containing, and mitigating the COVID-19 pandemic in the US. Response efforts involve the collection and analysis of data corresponding to healthcare organizations, public health departments, socioeconomic indicators, as well as additional signals collected directly from individuals and communities. We focused on electronic health record (EHR) data, since EHRs can be leveraged and scaled to improve clinical care, research, and to inform public health decision-making. We outline the current challenges in the data ecosystem and the technology infrastructure that are relevant to COVID-19, as witnessed in our 15 institutions. The infrastructure includes registries and clinical data networks to support population-level analyses. We propose a specific set of strategic next steps to increase interoperability, overall organization, and efficiencies.

OBJECTIVE: The goals of this study are to describe the value and impact of Project HealthDesign (PHD), a program of the Robert Wood Johnson Foundation that applied design thinking to personal health records, and to explore the applicability of the PHD model to another challenging translational informatics problem: the integration of AI into the healthcare system. MATERIALS AND METHODS: We assessed PHD's impact and value in 2 ways. First, we analyzed publication impact by calculating a PHD h-index and characterizing the professional domains of citing journals. Next, we surveyed and interviewed PHD grantees, expert consultants, and codirectors to assess the program's components and the potential future application of design thinking to artificial intelligence (AI) integration into healthcare. RESULTS: There was a total of 1171 unique citations to PHD-funded work (collective h-index of 25). Studies citing PHD span medical, legal, and computational journals. Participants stated that this project transformed their thinking, altered their career trajectory, and resulted in technology transfer into the commercial sector. Participants felt, in general, that the approach would be valuable in solving contemporary challenges integrating AI in healthcare including complex social questions, integrating knowledge from multiple domains, implementation, and governance. CONCLUSION: Design thinking is a systematic approach to problem-solving characterized by cooperation and collaboration. PHD generated significant impacts as measured by citations, reach, and overall effect on participants. PHD's design thinking methods are potentially useful to other work on cyber-physical systems, such as the use of AI in healthcare, to propose structural or policy-related changes that may affect adoption, value, and improvement of the care delivery system.

Objective: Understanding how to identify the social determinants of health from electronic health records (EHRs) could provide important insights to understand health or disease outcomes. We developed a methodology to capture 2 rare and severe social determinants of health, homelessness and adverse childhood experiences (ACEs), from a large EHR repository. Materials and Methods: We first constructed lexicons to capture homelessness and ACE phenotypic profiles. We employed word2vec and lexical associations to mine homelessness-related words. Next, using relevance feedback, we refined the 2 profiles with iterative searches over 100 million notes from the Vanderbilt EHR. Seven assessors manually reviewed the top-ranked results of 2544 patient visits relevant for homelessness and 1000 patients relevant for ACE. Results: word2vec yielded better performance (area under the precision-recall curve [AUPRC] of 0.94) than lexical associations (AUPRC = 0.83) for extracting homelessness-related words. A comparative study of searches for the 2 phenotypes revealed a higher performance achieved for homelessness (AUPRC = 0.95) than ACE (AUPRC = 0.79). A temporal analysis of the homeless population showed that the majority experienced chronic homelessness. Most ACE patients suffered sexual (70%) and/or physical (50.6%) abuse, with the top-ranked abuser keywords being "father" (21.8%) and "mother" (15.4%). Top prevalent associated conditions for homeless patients were lack of housing (62.8%) and tobacco use disorder (61.5%), while for ACE patients it was mental disorders (36.6%-47.6%). Conclusion: We provide an efficient solution for mining homelessness and ACE information from EHRs, which can facilitate large clinical and genetic studies of these social determinants of health.

OBJECTIVE: Personal health applications have the potential to help patients with chronic disease by improving medication adherence, self-efficacy, and quality of life. The goal of this study was to assess the impact of MyMediHealth (MMH) - a website and a short messaging service (SMS)-based reminder system - on medication adherence and perceived self-efficacy in adolescents with asthma. METHODS: We conducted a block-randomized controlled study in academic pediatric outpatient settings. There were 98 adolescents enrolled. Subjects who were randomized to use MMH were asked to create a medication schedule and receive SMS reminders at designated medication administration times for 3 weeks. Control subjects received action lists as a part of their usual care. Primary outcome measures included MMH usage patterns and self-reports of system usability, medication adherence, asthma control, self-efficacy, and quality of life. RESULTS: Eighty-nine subjects completed the study, of whom 46 were randomized to the intervention arm. Compared to controls, we found improvements in self-reported medication adherence (P = .011), quality of life (P = .037), and self-efficacy (P = .016). Subjects reported high satisfaction with MMH; however, the level of system usage varied widely, with lower use among African American patients. CONCLUSIONS: MMH was associated with improved medication adherence, perceived quality of life, and self-efficacy.Trial Registration This project was registered under http://clinicaltrials.gov/ identifier NCT01730235.

OBJECTIVE: Mobile health (mHealth) interventions may improve diabetes outcomes, but require engagement. Little is known about what factors impede engagement, so the authors examined the relationship between patient factors and engagement in an mHealth medication adherence promotion intervention for low-income adults with type 2 diabetes (T2DM). MATERIALS AND METHODS: Eighty patients with T2DM participated in a 3-month mHealth intervention called MEssaging for Diabetes that leveraged a mobile communications platform. Participants received daily text messages addressing and assessing medication adherence, and weekly interactive automated calls with adherence feedback and questions for problem solving. Longitudinal repeated measures analyses assessed the relationship between participants' baseline characteristics and the probability of engaging with texts and calls. RESULTS: On average, participants responded to 84.0% of texts and participated in 57.1% of calls. Compared to Whites, non-Whites had a 63% decreased relative odds (adjusted odds ratio [AOR] = 0.37, 95% confidence interval [CI], 0.19-0.73) of participating in calls. In addition, lower health literacy was associated with a decreased odds of participating in calls (AOR = 0.67, 95% CI, 0.46-0.99, P = .04), whereas older age (Pnonlinear = .01) and more depressive symptoms (AOR = 0.62, 95% CI, 0.38-1.02, P = .059) trended toward a decreased odds of responding to texts. CONCLUSIONS: Racial/ethnic minorities, older adults, and persons with lower health literacy or more depressive symptoms appeared to be the least engaged in a mHealth intervention. To facilitate equitable intervention impact, future research should identify and address factors interfering with mHealth engagement.

OBJECTIVE: To assess the state of readiness for the adoption of paperless labeling among a nationally representative sample of pharmacies, including chain pharmacies, independent retail pharmacies, hospitals, and other rural or urban dispensing sites. METHODS: Both quantitative and qualitative analyses were used to analyze responses to a cross-sectional survey disseminated to American Pharmacists Association pharmacists nationwide. The survey assessed factors related to pharmacists' attitudinal readiness (ie, perceptions of impact) and pharmacies' structural readiness (eg, availability of electronic resources, internet access) for the paperless labeling initiative. RESULTS: We received a total of 436 survey responses (6% response rate) from pharmacists representing 44 US states and territories. Across the spectrum of settings we studied, pharmacists had work access to computers, printers, fax machines and access to the internet or intranet. Approximately 79% of respondents believed that the initiative would improve the adequacy of drug information available in their work site and 95% believed it would either not change (33%) or would improve (62%) communication to patients. Overall, respondents' comments supported advancing the initiative; however, some comments revealed reservations regarding corporate or pharmacy buy-in, success of implementation, and ease of adoption. CONCLUSIONS: This is the first nationwide study to report about pharmacists' perspectives on paperless labeling. In general, pharmacists believe they are ready and that their pharmacies are well equipped for the transition to paperless labeling. Further exploration of perspectives from product label manufacturers and corporate pharmacy offices is needed to understand fully what will be necessary to complete this transition.

The increasing adoption of electronic health records (EHRs) and the corresponding interest in using these data for quality improvement and research have made it clear that the interpretation of narrative text contained in the records is a critical step. The biomedical literature is another important information source that can benefit from approaches requiring structuring of data contained in narrative text. For the first time, we dedicate an entire issue of JAMIA to biomedical natural language processing (NLP), a topic that has been among the most cited in this journal for the past few years. We start with a description of a contest to select the best performing algorithms for detection of temporal relationships in clinical documents (see page 806), followed by a general review of significance and brief description of commonly used methods to address this task (see page 814). Top performing approaches are featured in seven articles from five different countries—Canada (see page 843), China (see page 849), France (see page 820), Serbia (see page 859), and the US (see page 828, 836, 867).

OBJECTIVE: To examine the financial impact health information exchange (HIE) in emergency departments (EDs). MATERIALS AND METHODS: We studied all ED encounters over a 13-month period in which HIE data were accessed in all major emergency departments Memphis, Tennessee. HIE access encounter records were matched with similar encounter records without HIE access. Outcomes studied were ED-originated hospital admissions, admissions for observation, laboratory testing, head CT, body CT, ankle radiographs, chest radiographs, and echocardiograms. Our estimates employed generalized estimating equations for logistic regression models adjusted for admission type, length of stay, and Charlson co-morbidity index. Marginal probabilities were used to calculate changes in outcome variables and their financial consequences. RESULTS: HIE data were accessed in approximately 6.8% of ED visits across 12 EDs studied. In 11 EDs directly accessing HIE data only through a secure Web browser, access was associated with a decrease in hospital admissions (adjusted odds ratio (OR)=0.27; p<0001). In a 12th ED relying more on print summaries, HIE access was associated with a decrease in hospital admissions (OR=0.48; p<0001) and statistically significant decreases in head CT use, body CT use, and laboratory test ordering. DISCUSSION: Applied only to the study population, HIE access was associated with an annual cost savings of $1.9 million. Net of annual operating costs, HIE access reduced overall costs by $1.07 million. Hospital admission reductions accounted for 97.6% of total cost reductions. CONCLUSION: Access to additional clinical data through HIE in emergency department settings is associated with net societal saving.

OBJECTIVE: To evaluate the performance of a system that extracts medication information and administration-related actions from patient short message service (SMS) messages. DESIGN: Mobile technologies provide a platform for electronic patient-centered medication management. MyMediHealth (MMH) is a medication management system that includes a medication scheduler, a medication administration record, and a reminder engine that sends text messages to cell phones. The object of this work was to extend MMH to allow two-way interaction using mobile phone-based SMS technology. Unprompted text-message communication with patients using natural language could engage patients in their healthcare, but presents unique natural language processing challenges. The authors developed a new functional component of MMH, the Patient-centered Automated SMS Tagging Engine (PASTE). The PASTE web service uses natural language processing methods, custom lexicons, and existing knowledge sources to extract and tag medication information from patient text messages. MEASUREMENTS: A pilot evaluation of PASTE was completed using 130 medication messages anonymously submitted by 16 volunteers via a website. System output was compared with manually tagged messages. RESULTS: Verified medication names, medication terms, and action terms reached high F-measures of 91.3%, 94.7%, and 90.4%, respectively. The overall medication name F-measure was 79.8%, and the medication action term F-measure was 90%. CONCLUSION: Other studies have demonstrated systems that successfully extract medication information from clinical documents using semantic tagging, regular expression-based approaches, or a combination of both approaches. This evaluation demonstrates the feasibility of extracting medication information from patient-generated medication messages.

OBJECTIVE: The goal of this study was to develop an in-depth understanding of how a health information exchange (HIE) fits into clinical workflow at multiple clinical sites. MATERIALS AND METHODS: The ethnographic qualitative study was conducted over a 9-month period in six emergency departments (ED) and eight ambulatory clinics in Memphis, Tennessee, USA. Data were collected using direct observation, informal interviews during observation, and formal semi-structured interviews. The authors observed for over 180 h, during which providers used the exchange 130 times. RESULTS: HIE-related workflow was modeled for each ED site and ambulatory clinic group and substantial site-to-site workflow differences were identified. Common patterns in HIE-related workflow were also identified across all sites, leading to the development of two role-based workflow models: nurse based and physician based. The workflow elements framework was applied to the two role-based patterns. An in-depth description was developed of how providers integrated HIE into existing clinical workflow, including prompts for HIE use. DISCUSSION: Workflow differed substantially among sites, but two general role-based HIE usage models were identified. Although providers used HIE to improve continuity of patient care, patient-provider trust played a significant role. Types of information retrieved related to roles, with nurses seeking to retrieve recent hospitalization data and more open-ended usage by nurse practitioners and physicians. User and role-specific customization to accommodate differences in workflow and information needs may increase the adoption and use of HIE. CONCLUSION: Understanding end users' perspectives towards HIE technology is crucial to the long-term success of HIE. By applying qualitative methods, an in-depth understanding of HIE usage was developed.

OBJECTIVE: We assessed the usability of a health information exchange (HIE) in a densely populated metropolitan region. This grant-funded HIE had been deployed rapidly to address the imminent needs of the patient population and the need to draw wider participation from regional entities. DESIGN: We conducted a cross-sectional survey of individuals given access to the HIE at participating organizations and examined some of the usability and usage factors related to the technology acceptance model. MEASUREMENTS: We probed user perceptions using the Questionnaire for User Interaction Satisfaction, an author-generated Trust scale, and user characteristic questions (eg, age, weekly system usage time). RESULTS: Overall, users viewed the system favorably (ratings for all usability items were greater than neutral (one-sample Wilcoxon test, p<0.0014, Bonferroni-corrected for 35 tests). System usage was regressed on usability, trust, and demographic and user characteristic factors. Three usability factors were positively predictive of system usage: overall reactions (p<0 0.01), learning (p<0.05), and system functionality (p<0.01). Although trust is an important component in collaborative relationships, we did not find that user trust of other participating healthcare entities was significantly predictive of usage. An analysis of respondents' comments revealed ways to improve the HIE. CONCLUSION: We used a rapid deployment model to develop an HIE and found that perceptions of system usability were positive. We also found that system usage was predicted well by some aspects of usability. Results from this study suggest that a rapid development approach may serve as a viable model for developing usable HIEs serving communities with limited resources.

The Merriam-Webster dictionary defines ‘traction’ as the adhesive friction of a body on a surface on which it moves.1 Within the field of biomedical informatics, we have updated that definition so that the ‘body’ may refer to a technological advance, and the ‘surface’ to a person, group, or environment in which the technological advance has been introduced. In this context, traction implies not just adoption, but adherence, or the ‘state of steady or faithful attachment’. By any measure, the past 5 years has witnessed the attainment of traction by computerized provider order entry (CPOE). Certainly, the work undertaken by the Institute of Medicine to position CPOE as the most critical component of a safe decision-making environment,2–5 leading to the eventual mandates for CPOE as a part of certified health information technology,6 justifies this assertion. The early efforts of informatics researchers such as McDonald,7 Miller et al,8 Geissbuhler and Miller9 and Warner et al,10 who first described the potential of clinical decision support during order entry has finally been accepted. The evidence in support of this technology is, in fact, sufficiently compelling that there is no longer much value in publishing any but the most innovative and well-designed studies in this domain. Despite the attainment of traction, researchers in our field have not ignored many of the challenges associated with the vision of quality healthcare combined with usable tools. In the world of order entry decision-support, usable implies addressing the challenges of alert fatigue, high rates of alert overrides and human factors engineering to align the cognitive process of ordering with user interfaces for CPOE. There have been numerous reports about the unintended consequences of using CPOE. These reports range from an early observation about problems with picklists11 to adoption barriers12 and later, unintended consequences13–15 and even errors facilitated by CPOE.16 This issue of JAMIA includes a number of articles focused on these ongoing CPOE challenges. One of those challenges—the issue of alert fatigue—can be addressed through innovative human factors engineering. The articles led by Scott et al17 and Riedman et al18 specifically discuss this approach. Using simulation, Scott and colleagues17 compared the prescribing error rates associated with displaying electronic prescribing alerts using interrupting versus non-interrupting modalities. Their results address the relative advantages and disadvantages of different design strategies for commercial e-prescribing decision support, and remind us about the importance of human factors engineering expertise in clinical systems development. Riedman and colleagues18 report on a Delphi study to prioritize the best ways to improve medication delivery alerts. This study addresses some of the attributes of potential drug interactions that, if included in drug knowledge bases, could be exploited using human factors engineering to help decision makers respond to alerts. In addition to the alignment of functional requirements with good design principles, attention to prescribing workflow is an emerging area of importance. Baysari and colleagues19 report on their observational study involving teams of physicians on ward rounds, as they encountered prescribing alerts that should have potentially previously planned therapeutic interventions. Their paper builds on the early observations by many researchers in the field who have noted the importance of understanding established workflow as a prerequisite to system design. Articles in this issue also remind us about the need to measure rates of error and guideline adherence to improve CPOE systems iteratively. For example, Nanji and colleagues20 used a retrospective method to evaluate the incidence of medication prescribing errors after implementing an e-prescribing system. Their study identified a few categories of errors that may come as a surprise to some JAMIA readers. Wetterneck and colleagues21 evaluated the incidence of duplicate medication orders before and after CPOE implementation. Their study utilized a pre-post methodology and identified factors that led to a significantly higher rate of duplicate orders after CPOE implementation. Finally, the study by Austrian22 points to the importance of careful comparative effectiveness research methods to assess how CPOE can impact guideline adherence. Of course, as with all technological advances, it is clear that the traction provided by CPOE allows other technologies to evolve. In this issue, Cheung and colleagues23 continue a discussion catalyzed recently by Friedman et al24 about a learning e-health system. Cheung et al23 describe a registry for medication incidents that features multidisciplinary input and prescriber or pharmacy comparative feedback. The system also supports centralized surveillance for serious incidents, and mechanisms to disseminate warnings at that scale. Efforts to improve CPOE further, to integrate CPOE into the evolving landscape of health information technology and to propose breakthrough ideas in this domain are underway by experts in biomedical informatics. JAMIA will continue to be a source of these innovative and transformative articles. None. Commissioned; internally peer reviewed.

OBJECTIVE: Health information exchange (HIE) systems are being developed across the nation. Understanding approaches taken by existing successful exchanges can help new exchange efforts determine goals and plan implementations. The goal of this study was to explore characteristics of use and users of a successful regional HIE. DESIGN: We used a mixed-method analysis, consisting of cross-sectional audit log data, semi-structured interviews, and direct observation in a sample of emergency departments and ambulatory safety net clinics actively using HIE. For each site, we measured overall usage trends, user logon statistics, and data types accessed by users. We also assessed reasons for use and outcomes of use. RESULTS: Overall, users accessed HIE for 6.8% of all encounters, with higher rates of access for repeat visits, for patients with comorbidities, for patients known to have data in the exchange, and at sites providing HIE access to both nurses and physicians. Discharge summaries and test reports were the most frequently accessed data in the exchange. Providers consistently noted retrieving additional history, preventing repeat tests, comparing new results to retrieved results, and avoiding hospitalizations as a consequence of HIE access. CONCLUSION: HIE use in emergency departments and ambulatory clinics was focused on patients where missing information was believed to be present in the exchange and was related to factors including the roles of people with access, the setting, and other site-specific issues that impacted the overall breadth of routine system use. These data should form an important foundation as other sites embark upon HIE implementation.

Explicit guidelines are needed to develop safe and effective patient portals. This paper proposes general principles, policies, and procedures for patient portal functionality based on MyHealthAtVanderbilt (MHAV), a robust portal for Vanderbilt University Medical Center. We describe policies and procedures designed to govern popular portal functions, address common user concerns, and support adoption. We present the results of our approach as overall and function-specific usage data. Five years after implementation, MHAV has over 129,800 users; 45% have used bi-directional messaging; 52% have viewed test results and 45% have viewed other medical record data; 30% have accessed health education materials; 39% have scheduled appointments; and 29% have managed a medical bill. Our policies and procedures have supported widespread adoption and use of MHAV. We believe other healthcare organizations could employ our general guidelines and lessons learned to facilitate portal implementation and usage.

Clinical documentation is central to patient care. The success of electronic health record system adoption may depend on how well such systems support clinical documentation. A major goal of integrating clinical documentation into electronic heath record systems is to generate reusable data. As a result, there has been an emphasis on deploying computer-based documentation systems that prioritize direct structured documentation. Research has demonstrated that healthcare providers value different factors when writing clinical notes, such as narrative expressivity, amenability to the existing workflow, and usability. The authors explore the tension between expressivity and structured clinical documentation, review methods for obtaining reusable data from clinical notes, and recommend that healthcare providers be able to choose how to document patient care based on workflow and note content needs. When reusable data are needed from notes, providers can use structured documentation or rely on post-hoc text processing to produce structured data, as appropriate.

OBJECTIVE: To evaluate the impact of generic substitution decision support on electronic (e-) prescribing of generic medications. DESIGN: The authors analyzed retrospective outpatient e-prescribing data from an academic medical center and affiliated network for July 1, 2005-September 30, 2008 using an interrupted time-series design to assess the rate of generic prescribing before and after implementing generic substitution decision support. To assess background secular trends, e-prescribing was compared with a concurrent random sample of hand-generated prescriptions. MEASUREMENTS: Proportion of generic medications prescribed before and after the intervention, evaluated over time, and compared with a sample of prescriptions generated without e-prescribing. RESULTS: The proportion of generic medication prescriptions increased from 32.1% to 54.2% after the intervention (22.1% increase, 95% CI 21.9% to 22.3%), with no diminution in magnitude of improvement post-intervention. In the concurrent control group, increases in proportion of generic prescriptions (29.3% to 31.4% to 37.4% in the pre-intervention, post-intervention, and end-of-study periods, respectively) were not commensurate with the intervention. There was a larger change in generic prescribing rates among authorized prescribers (24.6%) than nurses (18.5%; adjusted OR 1.38, 95% CI 1.17 to 1.63). Two years after the intervention, the proportion of generic prescribing remained significantly higher for e-prescriptions (58.1%; 95% CI 57.5% to 58.7%) than for hand-generated prescriptions ordered at the same time (37.4%; 95% CI 34.9% to 39.9%) (p<0.0001). Generic prescribing increased significantly in every specialty. CONCLUSION: Implementation of generic substitution decision support was associated with dramatic and sustained improvements in the rate of outpatient generic e-prescribing across all specialties.

The objective of this review was to describe methods used to study and model workflow. The authors included studies set in a variety of industries using qualitative, quantitative and mixed methods. Of the 6221 matching abstracts, 127 articles were included in the final corpus. The authors collected data from each article on researcher perspective, study type, methods type, specific methods, approaches to evaluating quality of results, definition of workflow and dependent variables. Ethnographic observation and interviews were the most frequently used methods. Long study durations revealed the large time commitment required for descriptive workflow research. The most frequently discussed technique for evaluating quality of study results was triangulation. The definition of the term "workflow" and choice of methods for studying workflow varied widely across research areas and researcher perspectives. The authors developed a conceptual framework of workflow-related terminology for use in future research and present this model for use by other researchers.

Medication information is one of the most important types of clinical data in electronic medical records. It is critical for healthcare safety and quality, as well as for clinical research that uses electronic medical record data. However, medication data are often recorded in clinical notes as free-text. As such, they are not accessible to other computerized applications that rely on coded data. We describe a new natural language processing system (MedEx), which extracts medication information from clinical notes. MedEx was initially developed using discharge summaries. An evaluation using a data set of 50 discharge summaries showed it performed well on identifying not only drug names (F-measure 93.2%), but also signature information, such as strength, route, and frequency, with F-measures of 94.5%, 93.9%, and 96.0% respectively. We then applied MedEx unchanged to outpatient clinic visit notes. It performed similarly with F-measures over 90% on a set of 25 clinic visit notes.

OBJECTIVE: Clinical notes, typically written in natural language, often contain substructure that divides them into sections, such as "History of Present Illness" or "Family Medical History." The authors designed and evaluated an algorithm ("SecTag") to identify both labeled and unlabeled (implied) note section headers in "history and physical examination" documents ("H&P notes"). DESIGN: The SecTag algorithm uses a combination of natural language processing techniques, word variant recognition with spelling correction, terminology-based rules, and naive Bayesian scoring methods to identify note section headers. Eleven physicians evaluated SecTag's performance on 319 randomly chosen H&P notes. MEASUREMENTS: The primary outcomes were the algorithm's recall and precision in identifying all document sections and a predefined list of twenty-nine major sections. A secondary outcome was to evaluate the algorithm's ability to recognize the correct start and end boundaries of identified sections. RESULTS: The SecTag algorithm identified 16,036 total sections and 7,858 major sections. Physician evaluators classified 15,329 as true positives and identified 160 sections omitted by SecTag. The recall and precision of the SecTag algorithm were 99.0 and 95.6% for all sections, 98.6 and 96.2% for major sections, and 96.6 and 86.8% for unlabeled sections. The algorithm determined the correct starting and ending text boundaries for 94.8% of labeled sections and 85.9% of unlabeled sections. CONCLUSIONS: The SecTag algorithm accurately identified both labeled and unlabeled sections in history and physical documents. This type of algorithm may assist in natural language processing applications, such as clinical decision support systems or competency assessment for medical trainees.

OBJECTIVES: The goal of the study was to develop an in-depth understanding of work practices, workflow, and information flow in chronic disease care, to facilitate development of context-appropriate informatics tools. DESIGN: The study was conducted over a 10-month period in three ambulatory clinics providing chronic disease care. The authors iteratively collected data using direct observation and semi-structured interviews. MEASUREMENTS: The authors observed all aspects of care in three different chronic disease clinics for over 150 hours, including 157 patient-provider interactions. Observation focused on interactions among people, processes, and technology. Observation data were analyzed through an open coding approach. The authors then developed models of workflow and information flow using Hierarchical Task Analysis and Soft Systems Methodology. The authors also conducted nine semi-structured interviews to confirm and refine the models. RESULTS: The study had three primary outcomes: models of workflow for each clinic, models of information flow for each clinic, and an in-depth description of work practices and the role of health information technology (HIT) in the clinics. The authors identified gaps between the existing HIT functionality and the needs of chronic disease providers. CONCLUSIONS: In response to the analysis of workflow and information flow, the authors developed ten guidelines for design of HIT to support chronic disease care, including recommendations to pursue modular approaches to design that would support disease-specific needs. The study demonstrates the importance of evaluating workflow and information flow in HIT design and implementation.

OBJECTIVE: Evaluations of individual terminology systems should be driven in part by the intended usages of such systems. Clinical interface terminologies support interactions between healthcare providers and computer-based applications. They aid practitioners in converting clinical "free text" thoughts into the structured, formal data representations used internally by application programs. Interface terminologies also serve the important role of presenting existing stored, encoded data to end users in human-understandable and actionable formats. The authors present a model for evaluating functional utility of interface terminologies based on these intended uses. DESIGN: Specific parameters defined in the manuscript comprise the metrics for the evaluation model. MEASUREMENTS: Parameters include concept accuracy, term expressivity, degree of semantic consistency for term construction and selection, adequacy of assertional knowledge supporting concepts, degree of complexity of pre-coordinated concepts, and the "human readability" of the terminology. The fundamental metric is how well the interface terminology performs in supporting correct, complete, and efficient data encoding or review by humans. RESULTS: Authors provide examples demonstrating performance of the proposed evaluation model in selected instances. CONCLUSION: A formal evaluation model will permit investigators to evaluate interface terminologies using a consistent and principled approach. Terminology developers and evaluators can apply the proposed model to identify areas for improving interface terminologies.

To facilitate the future implementation of an electronic prescribing system, this case study modeled prescription management processes in various primary care settings. The Vanderbilt e-prescribing design team conducted initial interviews with clinic managers, physicians and nurses, and then represented the sequences of steps carried out to complete prescriptions in activity diagrams. The diagrams covered outpatient prescribing for patients during a clinic visit and between clinic visits. Practice size, practice setting, and practice specialty type influenced the prescribing processes used. The model developed may be useful to others engaged in building or tailoring an e-prescribing system to meet the specific workflows of various clinic settings.

Previous investigators have defined clinical interface terminology as a systematic collection of health care-related phrases (terms) that supports clinicians' entry of patient-related information into computer programs, such as clinical "note capture" and decision support tools. Interface terminologies also can facilitate display of computer-stored patient information to clinician-users. Interface terminologies "interface" between clinicians' own unfettered, colloquial conceptualizations of patient descriptors and the more structured, coded internal data elements used by specific health care application programs. The intended uses of a terminology determine its conceptual underpinnings, structure, and content. As a result, the desiderata for interface terminologies differ from desiderata for health care-related terminologies used for storage (e.g., SNOMED-CT), information retrieval (e.g., MeSH), and classification (e.g., ICD9-CM). Necessary but not sufficient attributes for an interface terminology include adequate synonym coverage, presence of relevant assertional knowledge, and a balance between pre- and post-coordination. To place interface terminologies in context, this article reviews historical goals and challenges of clinical terminology development in general and then focuses on the unique features of interface terminologies.

Electronic prescribing (e-prescribing) systems can provide computer-based support for the creation, transmission, dispensing, and monitoring of pharmacological therapies. In the United States and other countries, such systems have been documented, under certain conditions, to increase the safety and quality of patient care.1–5 The authors applaud the initial efforts of Teich and colleagues in the Joint Clinical Decision Support Workgroup (Joint CDS WG) to outline e-prescribing desiderata, as reported in this issue of JAMIA by Teich et al.6 Their article is published as an endorsed policy of the American Medical Informatics Association (AMIA). Previously, Bell et al. published an excellent list of desiderata for outpatient e-prescribing and sorted the desiderata into functional categories.7 Subsequently, Wang et al. surveyed e-prescribing vendor systems to determine that existing systems on average met only half the desiderata, with none exceeding 64% fulfillment.8 The recommendations outlined in the tables of the Joint CDS WG provide a useful point of departure for future discussions. Of note, the Joint CDS WG guidelines were developed as a “commissioned work” with externally determined foci, time limitations, and priorities, so that those guidelines do not fully cover all relevant areas. The Joint CDS WG document therefore represents an important first step in an evolving approach to a complex set of problems. The Joint CDS WG recommendations present a scenario of how e-prescribing features might be rolled out. The authors of this commentary would like to supplement, from what we believe is a broader perspective, the focused set of Joint CDS WG recommendations. The Joint CDS WG proposal has several strengths, including the recommendations that the United States should develop and promote shareable standards for e-prescribing and related decision support systems, a consensus should be developed on how to implement and evaluate decision support systems, and certain organizations, (such as the Office of the National Coordinator for Health Information Technology, the Agency for Healthcare Research and Quality, the U.S. Food and Drug Administration (FDA), the National Library of Medicine, AMIA, the e-Health Initiative, and the Health Information and Management Systems Society) should take leadership roles in the e-prescribing efforts. The authors note that developers and implementers should consider the variability that currently exists among users, clinical settings, information systems, and environments when determining how and when to install and support an e-prescribing system. For example, even when clinical systems provide net benefits to an institution, the implementation of electronic systems to improve the quality of care can introduce unwanted, potentially harmful side effects that must be detected, monitored, and addressed.9,10 It is therefore important to consider the potential adverse effects of e-prescribing implementation. Electronic prescribing systems represent only one genre of electronic health record system activity (others include departmental pharmacy, radiology, and laboratory systems and systems for record keeping, ordering, results display, monitoring, and decision support). Because the current state of the art for complex, comprehensive electronic health record systems is immature,11 there is not yet a scientific basis for selecting among the many potential courses of action related to implementation and use of e-prescribing systems. It is the authors' opinion that human (end-user) factors and electronic information interchanges among e-prescribing and other clinical systems play critically important roles in determining the success or failure of e-prescribing systems. These considerations should be combined with the Joint CDS WG suggestions when making implementation decisions. Several articles in the current issue of JAMIA illustrate how intricate and difficult it is to implement and evaluate such systems. There are few operational systems in place that have documented the success or failure of e-prescribing guidelines outlined. The authors note that e-prescribing systems alone may not suffice; more comprehensive electronic health systems may be required to address the needs of both healthcare facilities and individual practitioners. Clinicians should be wary of developing a false sense of security and unrealistic expectations based on use of e-prescribing applications alone, when more complex systems may be required. To reap the benefits of several decades of dedicated work by biomedical informaticians, commercial vendors, and health care providers (institutions and individuals), a responsible approach to e-prescribing must be advocated. All parties with a stake in e-prescribing must develop a common, overarching framework for its development and dissemination. The remainder of this commentary examines the environmental factors, technical factors, and strategic factors relevant to e-prescribing before concluding with a recommended framework that builds on and supplements the Joint CDS WG e-prescribing recommendations. Environmental factors relevant to e-prescribing include individual practitioners' specialties and roles; the variety of practice settings in which care is delivered in the United States; the standard of care in the clinical community; and end-user constraints imposed by human limitations in knowledge, habits, and work flows. Pragmatically, clinicians' individual practice circumstances should determine their access to, choice of, and use of e-prescribing technology. Currently in the United States, the status of e-prescribing systems varies by geographic regions and by federal, state, or local governmental jurisdictions; practice setting/type; and commercial vendor application. As a result, there are widely varying e-prescribing adoption rates. It might be argued that in today's health care environment, a rural general practitioner who manages inpatients in the morning, outpatients in a small office in the afternoon, and who makes house calls as needed cannot and should not use the same e-prescribing system in each setting, even though in the future one system should suffice. Today's systems with one underlying knowledge base cannot easily switch between inpatient and outpatient formularies (which differ significantly). The rich information environment of hospital-based electronic medical record and computerized provider order entry (CPOE) systems is not easily replicated in outpatients' homes, even with remote wireless connectivity. Large academic medical centers have the resources (adequate teams of talented informaticians, available expert clinicians, and an annual budget of millions of dollars to support their work) to develop or purchase, customize, roll out, and evolve state-of-the-art e-prescribing systems. By contrast, both solo providers and small rural hospitals have limited access to informatics expertise, and little time or money to develop or install complex systems.12,13 In the inpatient setting, when a clinician orders a medication, there is typically only one inpatient pharmacy system through which the order will be processed, and it is the same system for all providers and all orders. In the outpatient setting, patients typically receive multiple prescriptions from multiple care providers and may fill them at different pharmacies. Each retail pharmacy store (or pharmacy chain) may have its own software system that provides various levels of alerts regarding doses and drug interactions to pharmacists as they fill prescriptions, but the same prescription taken to different pharmacies will generate different alerts. Electronic connectivity is rare between free-standing outpatient pharmacies and the hospital or clinic-based, patient information–rich practice settings where providers generate prescriptions. Most connectivity that exists takes the form of fax machines, which, it is hoped, produce legible prescriptions but which still do not preclude transcription (and other) errors. For example, it would not be uncommon for a physician to write, “warfarin 5 mg one tablet by mouth daily” and the pharmacist to dispense (due to a temporary shortage) 2.5 mg tablets, with the instruction “take two tablets daily.” When in the following week, the physician receives a subtherapeutic international normalized ratio (INR) blood test result for the patient (indicating that the dose should be increased), a phone call to the patient's home to “take one and one-half pills daily” may have disastrous consequences. The patient would take a decreased dose (3.75 mg) instead of the physician's intended dose (7.5 mg) due to failed communication among systems, providers, and the patient. “Closed loop” e-prescribing feedback that matches clinicians' orders with pharmacy dispensing annotations requires a bidirectional interface from prescribing site to the dispensing pharmacy. What is required is that the systems “match up” the dispensing record with the prescribing order in a manner in which any clinician reviewing the patient's chart could easily determine what was ordered and how it was dispensed, so as to avoid the previous scenario. The few bidirectional interfaces now in existence often generate a request for the attending physician's countersignature whenever the pharmacist changes the dispensing record. The latter model is unworkable in terms of introducing an unnecessary and often confusing extra burden on the physician, who may not understand which medication order is being changed for what dispensing reason if multiple changes are made. Clinicians who are told to use an e-prescribing system cannot be expected to do so if using the system compromises the clinicians' existing standard of care for patients. Weight-based dosing represents the standard of care for many medications and ages in the pediatric population. A 1998 article in Pediatrics titled “Prevention of Medication Errors in the Pediatric Inpatient Setting” listed “confirm that the patient's weight is correct for weight-based dosages” as its first recommendation under “medication ordering to reduce errors.”14 In a related article in 2004, Kaushal et al. cited error-prone behaviors in clinicians' manual calculations of weight-based pediatric medication dosages.15 The authors believe that weight-based dose calculations for pediatric patients is an established standard of practice in the community that should be adopted within e-prescribing systems immediately (i.e., by 2006), even though many current vendor pharmacy and CPOE products poorly support pediatric dosing (and especially dosing in premature neonates). A number of academic centers have demonstrated that pediatric dosing can be done as part of electronic prescribing. Those centers have developed, deployed, and evaluated reliable pediatric and neonatal dosing systems over the In the current issue of et present a of a that potentially in part by use of an inpatient e-prescribing system. the of complex interactions among system and environmental factors that can to whenever and feedback Systems that clinician by not all relevant information for decision making into one place the of the clinician must still the record (or a clinical results as as with an e-prescribing system the result can be more work and for the clinician as as more to by important a or so with can a of errors. The in this issue by and provides a of how systems that might be can effects when in the clinical environment due to a of failure of to the system clinical of the environment, and of various among different electronic systems, among clinicians, or between and systems can to errors. must be to the environment as as to the systems being In the current issue of the by et al. the factors that clinician with and that environmental factors can play roles in or to use of between and using the not with the implementation of recommended of clinical recommendation and interface the number of at a strategic of the of into and the to document system and receive In previous of et al. the human factors required for implementation of clinical and and argued that the important implementation are factors relevant to e-prescribing systems include the information of a the for system the development for the system the to which clinician in feedback at the implementation and use the electronic interface of the system to other electronic the interface of the the of a combined who and system the of local to have into system and the to changes within to not or and and for quality both and developers of such systems. an base exists in the under e-prescribing systems can improve patient safety and quality of the under which such cannot be easily Most the of CPOE and electronic medical record systems have from academic medical centers that the resources and both informatics and to and clinical decision support the information knowledge underlying commercial CPOE and pharmacy system vendor products are typically by commercial (such as and and to CPOE The quality and of pharmacy information have been and their has at been into The efforts reported by et are of this the success reported by that such when can be The and safety of e-prescribing systems on their underlying pharmacy information the of the and the circumstances of software and system implementation. In the systems available to the of hospitals and in the United States were developed by commercial vendors, systems typically the state of the art in the academic centers by several For example, that vendor systems are in the of pediatric prescribing. The authors are of a number of of hospitals pharmacy systems and or the pediatric prescribing due to of by the systems. of one commercial drug information system in an e-prescribing (such as dose of in and for and that use of the system local was The of and the of with many commercial vendor systems (and are not to the Most commercial pharmacy information systems generate a (and number of drug et al. a for all based on a failure of the to how often interactions in patient and the of those when they As et al. note, often at the physician to in the et al. the of resources that must be dedicated to and the commercial drug information that must be at medical that such systems. The of at each local medical the are both and at local is required at system desiderata, including use of were in the of Bell et the Joint CDS WG et and in the for system developers outlined by and In to desiderata, it is to consider and when e-prescribing systems. For example, information should be from only when the information will be for important decisions. the should be only from the individual to the correct a clinician in patient or laboratory results so that the information can be as or on a prescription may be the e-prescribing system should decision support that laboratory results or if to provide dosing recommendations and The to e-prescribing system reviewing alerts and taken is for quality in As by and in this issue of such as may the of for use by decision support systems and may be to of e-prescribing of factors relevant to e-prescribing include of or how to e-prescribing systems through published related to the of e-prescribing system (such as governmental and Joint on of Healthcare related to and and such as development of standards monitoring of an e-prescribing system over time on the quality and of the knowledge base underlying the e-prescribing system at any point in the quality and of the software system the knowledge base to a patient's clinical for prescribing the and for of both the knowledge base and the The Joint CDS WG that drug information should be if a number of such are of one and if their knowledge and terms of quality of and over they might not to the quality of drug information systems. The of for e-prescribing systems is an but we cannot how each of the to by the Joint CDS WG will to it is important to of the underlying drug information knowledge base from of the related software cannot for quality drug and an quality knowledge base may be by can of one so that a system as may from over time if the knowledge base is not or if software include The authors believe that it is to if will work or if the for Healthcare Information is the for such can be demonstrated to be and The authors believe that a framework should be developed and to promote adoption of e-prescribing systems in outline the that such a framework might both now and in the system must improve over time in a and are limited to e-prescribing systems and their and do not address more comprehensive medication safety such as of both patients and medications and of medication orders and doses at the point of care to A approach to what has been and will be through e-prescribing is to making Electronic prescribing cannot an activity is developed for its in settings academic cannot be in alone should not patients e-prescribing system that is the patient information from an system can produce clinical should in The governmental and should the development of standards for transmission, and monitoring of information and play a in the development and of quality drug information knowledge The United States has taken a through in developing standards for the prescribing and dispensing of The and related and Drug products represent excellent A step would be to develop consensus standards for e-prescribing information For example, a is needed for and drug including the and of standards for the and of interactions within drug information should be each vendor its own systems for such information and only this to an of e-prescribing as reported by et in this but it potentially in which, for example, a CPOE system cannot to pharmacy system (or what is to be potentially for a patient the CPOE system might as to but the pharmacy system might not be to the (or A standard for drug information (or for adverse effects of individual should have an it might for example, the following of the that (or of the drug with an adverse a but standard set of for the clinical of the interactions A the of drug A and drug or A of drug an a of the of the base for the based on in or to based on knowledge of but reported in reported as or reported as a of done replicated by multiple a for the of but or potential for or or potentially or and to in a on a of how often each has been reported to for example, such a might 5 more of to of one in to in to such an standard set of for drug adverse effects and drug e-prescribing systems could more determine how to the of alerts to the for and potentially more alerts in a more manner alerts. There are a number of academic and commercial developers who now have from developing e-prescribing systems. such as up” information (such as current patient or with a or when the is to a if the system laboratory results medication ordering (such as and levels when an is it is important to the and time that the laboratory results were a physician might and as being current when ordering today's when in the laboratory results are (and might if the clinician In the current issue of et al. that the same delivered through different interface can to different by their interface to a for drug information The should develop a that on the of et and the general recommendations of et to useful information for future developers and for to install e-prescribing systems. and academic centers in should development of e-prescribing standards as for both information and consider their drug information knowledge to be and and as a result, they are not to an of by In the current environment, e-prescribing systems may not even have access to the drug information knowledge underlying the systems, making quality a In an both their own systems with such and the same systems with the same would produce the same or can and of for drug information knowledge for e-prescribing system software might drug information knowledge to a should that the clinical standard of care in the such as weight-based dosing for is in e-prescribing systems. When it is should be and system under such circumstances might be if weight-based dosing is not the system could not generate prescriptions for should be developed that provide to which vendor and systems have been evaluated at what in time the to the results or academic medical centers should develop based on of vendor (or e-prescribing systems that are available for which of systems or system features (such as the desiderata of the Joint CDS WG) are recommended for in various A approach might be might the current of practice environment on a that with of the would the of the e-prescribing a solo practitioner in an a small a a practice or with a to medical a small a a hospital and a academic medical would various of systems (or system as or and the resources in and information required to use recommendations for e-prescribing systems would with each so that a solo practitioner with e-prescribing system now might be to one of a number of computer-based e-prescribing systems and to only use it for prescribing when needed as to not practice but to provide information and in any By contrast, for a academic medical with an electronic medical record the might of a e-prescribing approach and relevant to use in selecting such a system. organizations, and academic medical centers should promote and for adverse with a system for is an important first the authors that only the has the and resources to the of to a common, of drug information to support e-prescribing over efforts to as as the National Library of to and the Medical can as for how a drug information knowledge base could be The has a among governmental academic medical and commercial to develop and a useful the such an the drug information of medications might include recommended doses for each medication dose form by and and interactions with and of quality of number of of the and other and The drug information knowledge base could be to and commercial In the of a drug information knowledge developers and should both the drug information underlying their e-prescribing products and the related software to for and quality (which will should through and academic centers and to develop the to such and develop standard for the should the results of of their products they are must evolve the and at local to drug information making them more to and to from the should in any governmental to a drug information knowledge base recommended now developing and drug information knowledge and e-prescribing systems not of if the at future provides a drug information knowledge base for all to The could the knowledge base at a and by both quality information resources and useful e-prescribing software with decision support through such as the Health Information and Management Systems should be to develop a system that from and the to provide to their systems on a or or basis to improve patient A set of standards to should be developed for the e-prescribing that e-prescribing systems have the to products and to feedback for quality must determine that the burden of and using e-prescribing systems not the quality of even if the quality of prescribing is For example, if the of using e-prescribing is for each practitioner to patients the net of an excellent system may be individual care providers must first to use e-prescribing systems, must and must provide feedback for quality By patients that the provider is using an e-prescribing the provider the and or use of and the provider alerts the patient in the patient is by a which to be an of a The time for developing a approach to e-prescribing is at The desiderata by the Joint CDS WG present an important step this list of desiderata is it is only a first should be developed over time to for among users, settings, and information systems. In the authors note that individual e-prescribing recommendations may not be for all settings pediatric on of the recommendations are and will to improve the guidelines over is including the of effects of such systems, with development of reliable to in a The authors believe that of such systems will be complex and we the and of that patient safety cannot be done in an manner or to and correct as they work will be required through of governmental academic and The authors applaud the Joint CDS WG recommendations as a step that will evolve based on feedback and in

Mohammad Al-Ubaydli, MD, a Visiting Research Fellow at the National Center for Biotechnology Information (NCBI) in Washington, DC, recently has published an information guide entitled Handheld Computers for Doctors. A handheld computer, also known as a personal digital assistant (PDA), is defined in this report as “a computer that is small enough to hold in your hand, or keep in your pocket.” This book addresses four main topics: What is a handheld computer? What utility does it provide for personal organization and clinical information management? What are some practical applications of networked handhelds in the clinical setting? How does an individual become a successful handheld computer project champion? The book is arranged into three sections that are well organized to address these questions. The first section, termed Why Star Trek Is Science Past, is clearly targeted at clinicians who are skeptical about the technology or who recently became new users of handheld computers. Dr. Al-Ubaydli provides insight into choosing basic hardware and software, and covers both Palm and PocketPC handheld computer environments. Some of the general functions covered include personal organization (calendar, address book, and notepad), Internet access, electronic book options, and, of course, games. The section also addresses basic tools available on handheld computers for clinical information management, including structured outline tools for scientific lectures, patient data management capabilities, and medical reference documents.

Peer review is defined as “an evaluation by experts of the quality and pertinence of research or research proposals of other experts in the same field.”1 Peer review is a key component of the process by which an academic journal retains its quality and scientific rigor. Peer review serves three main purposes: providing a quasi-objective metric for the quality of journal submissions, serving as a mechanism to improve the quality of the content, and providing a mechanism for informing and educating journal contributors (authors). Reviewers help editors to determine whether a manuscript is worthy of publication in the specific journal. Not only do reviewers give their opinion of the merits of the manuscript on a “publishability” scale, they explain to the Editorial Office why they ranked the submission as they did and provide suggestions for how to improve the manuscript to both editors and authors. This process allows a discipline to maintain and improve the quality of its published papers. This process secondarily affects what research is conducted, what methodologies are employed, and what messages are disseminated to professional and lay audiences and, to some extent, rightly or wrongly, provides a mechanism to assist in evaluating the work of the authors who submit publications. Peer review is at first glance a thankless job for the anonymous reviewer. It takes time and effort to do well, for which the primary reward is the contribution the reviewer makes to the academic and professional communities served by the journal. However, one gains skill and knowledge from participating in the peer-review process. Reviewers learn effective methods for organizing and presenting scientific content from the authors who submit manuscripts. They also learn from the scientific ideas in the manuscripts they review (even though they cannot act on them or discuss them with others until the manuscript is published in the public domain). Reviewers gain knowledge, understanding, and perspective from reading the confidential reviews (distributed by many journals, including JAMIA, among all reviewers) of other peer reviewers who have submitted opinions regarding the same manuscript. Reviewers also learn from observing the process through which authors iteratively respond to (or fail to respond to) critiques in successive revisions of a manuscript. Despite the longstanding tradition of this approach to manuscript quality control, the medical literature is replete with concerns about the peer-review process. A 1994 survey of authors submitting papers to Journal of Clinical Anesthesia noted that unclear comments, judgmental reviews, discrepant reviews, and disorganized management of the peer review process (including timely return of manuscripts and selection of knowledgeable reviewers) were primary reasons for dissatisfaction.2 These concerns are not limited to specific journals; in fact, prescriptions for constructing an acceptable review have been published for many leading journals.3–5 Given the importance of this task, and the widespread need for peer reviewers in all scientific disciplines, it would seem prudent to incorporate formal training about peer review into the training of academic professionals.6 In fact, some journals, such as the Annals of Emergency Medicine,5 have established training programs for new reviewers. It is of interest, however, that training and guidelines generally are provided after a reviewer has been given peer-review responsibility. Ideally, these practices would be more valuable if provided to new reviewers before they participate in official reviews, rather than through “on-the-job” training. This year, through the combined efforts of its editor, associate editors, and assistant editor, JAMIA designed and implemented a Student Editorial Board (SEB). This board was formed by selecting outstanding trainees who applied from the National Library of Medicine Training Program sites. Each program was invited to submit the names of up to two trainees to the Assistant Editor. Applicants were all well qualified, and had interests and experience spanning the breadth of biomedical informatics. After discussion and voting, the Associate Editors selected six trainees to form the inaugural Student Editorial Board. The current SEB members are: Tricia A. Thornton, BA (Vanderbilt University) Michael F. Chiang, MD (Columbia University College of Physicians and Surgeons) Peter Mork, MS (University of Washington, Seattle) Adam Rothschild, MD (Johns Hopkins University School of Medicine) Roderick Y. Son, MS (University of California, Los Angeles) Lisa J. Trigg, MN, ARNP (University of Washington, Seattle) The SEB is designed to provide hands-on experience to prospective biomedical informatics journal reviewers. We envision that this experience will serve a variety of purposes. First and foremost, the SEB experience will allow prospective reviewers to learn how to construct a well-written, critical, and constructive review. To accomplish this objective, each SEB member is given a monograph written by the JAMIA Editor and Assistant Editor, citing previous work of Stead et al.,7 Friedman,8 and Aydin.9 This monograph describes the elements of a good review, the ethical responsibilities of peer reviewers, and the generally acceptable tone and style of a critical review. Each time an SEB member completes a review, the lessons contained within the monograph are reinforced by a careful critique of each SEB review by the Assistant Editor . In addition, SEB members receive copies of the other JAMIA-assigned reviewers' comments once the editorial process has been completed on the submitted manuscript they reviewed. The SEB experience also will help SEB members understand how to publish work done at any stage of an informatics project. We will ask SEB members to review all types of materials submitted to JAMIA—including books, original research articles, viewpoint documents, model formulation papers, and case reports. We also will provide SEB members with revised and resubmitted manuscripts, so that they might see how a paper evolves from its first reviewed form all the way through to its final draft. Finally, we expect SEB members to learn the time management skills necessary to incorporate ad hoc, but time-critical work into their busy schedules, so that they do not find the process as onerous during their early years as independent investigators. Members will have an opportunity to complete six reviews (provided at times when their own publications or presentations should not compete for their attention). They are expected to adhere to the timeline provided to other reviewers. We will accomplish these objectives by integrating SEB members into the existing JAMIA review process. Each SEB member has provided the JAMIA Assistant Editor his or her areas of interest as a part of the application process. Students are expected to construct a timely, professional review of the manuscript with the same turnaround as regular reviewers. Student reviews are returned to the JAMIA office, where the Assistant Editor reviews them, provides constructive feedback, and accepts revisions by the SEB member. Once approved, the JAMIA Editor includes the student reviews as a part of the overall critique of the manuscript returned to authors. A label informs the author that the review is from an anonymous SEB member. The student reviews are intended to complement the reviews by JAMIA Editorial Board members and outside reviewers that will continue to occur. Students also have an opportunity to turn back manuscripts (not review them) when the timing of a JAMIA request is not convenient for them. Student Editorial Board members—like other editorial board members—are not financially compensated for their work. The major benefits to SEB members are education about the publication process and the construction of a useful review, as well as the notoriety from being selected. Student Editorial Board members also are recognized through a listing in JAMIA on the same page/e-location where the Editorial Board membership is listed. Thus far, the SEB process has proceeded smoothly. Our six SEB members recently completed their first set of reviews. A careful critique of these reviews reinforced the potential impact that the SEB will have on the written critiquing skills of its members. For example, critiques of these reviews were able to use specific textual examples to teach: The fine line between providing constructive insight to the author and clouding insight with condescending statements The importance of including an evaluation of prior work in the introduction of an original research manuscript The impact of poor author writing style on the ability of a reviewer to critique a potentially groundbreaking research project How the author's receptivity toward a review may be affected by the style of the review (particularly when the review addresses the author as “you”) Although it is too early to determine whether this process makes a difference in either the quality of our reviews or the skills of potential reviewers, comments from current SEB members suggest that the process is both enjoyable and instructive. They have been uniformly responsive to critiques of their reviews, and have made revisions that have generally produced a much more effective review. We intend to evaluate the effects of the SEB process more carefully over the year. We will soon expand the SEB from six to 12 members, with a plan to provide each SEB member with at least six reviews during his or her two-year tenure on the Board. The next round of applications will extend beyond trainees in U.S. training programs. In the meantime, we will evolve the SEB to meet the needs of our initial cohort of members, while striving to keep the process both fun and educational.

D. Curtis Jamison, PhD, an associate professor in the School of Computational Science at George Mason University in Manassas, Virginia, wrote the book Perl Programming for Biologists. Perl is an open-source interpreted scripting language originally designed for Unix systems programming by Larry Wall about 25 years ago. Due in large part to its powerful regular expressions for pattern matching and its accessibility for nonprogrammers, Perl has become the most popular programming language in bioinformatics. The goal of this book is to provide the reader with the background and tools necessary to write scripts of immediate practical value to his or her work. It is Dr. Jamison's long-held desire to see full integration of computer technology into the experimental protocol that he brings to Perl Programming for Biologists. That desire translates into a very accessible text, which any biomedical researcher lacking programming experience would find a quick and painless primer. This book clearly targets that audience, and in so doing is too elementary for most researchers with intermediate or advanced programming experience. Furthermore, most beginning programmers will find that as their programming projects grow more complex, they will soon need a more thorough treatment of the subject, such as that found in O'Reilly's Beginning Perl for Bioinformatics.1

OBJECTIVES: Web-based applications have been developed that allow patients to enter their own information into secure personal health records. These applications are being promoted as a means of providing patients and providers with universal access to updated medical information. The authors evaluated the functionality and utility of a selection of personal health records. DESIGN: A targeted search strategy was used to identify eleven Web sites promoting different personal health records. Specific criteria related to the entry and display of data elements were developed to evaluate the functionality of each PHR. Information abstracted from an actual case was used to create a series of representative PHRs. Output generated for review was evaluated to assess the accuracy and completeness of clinical information related to the diagnosis and treatment of specific disorders. RESULTS: The PHRs selected for review employed data entry methods that limited the range and content of patient-entered information related to medical history, medications, laboratory tests, diagnostic studies, and immunizations. Representative PHRs created with information abstracted from an actual case displayed varying amounts of information at basic and comprehensive levels of representation. CONCLUSIONS: Currently available PHRs demonstrate limited functionality. The data entry, validation, and information display methods they employ may limit their utility as representations of medical information.