OBJECTIVE: Accurate, complete allergy histories are critical for decision-making and medication prescription. However, allergy information is often spread across the electronic health record (EHR); thus, allergy lists are often inaccurate or incomplete. Discrepant allergy information can lead to suboptimal or unsafe clinical care and contribute to alert fatigue. We developed an allergy reconciliation module within Mass General Brigham (MGB)'s EHR to support accurate and intuitive reconciliation of discrepancies in the allergy list, thereby enhancing patient safety. MATERIALS AND METHODS: We combined data-driven methods and knowledge from domain experts to develop 5 mechanisms to compare allergy information across the EHR and designed a user interface to display discrepancies and suggested reconciliation actions, with links to relevant data sources. Qualitative and quantitative analyses were conducted to assess the module's performance and measure user acceptance. RESULTS: We implemented and tested the proposed allergy reconciliation mechanisms and module. A comprehensive integration workflow was developed for the module, which was piloted among 111 primary care physicians at MGB. F1 scores of the reconciliation mechanisms range from 0.86 to 1.0. Qualitative analysis showed majority positive feedback from pilot users. DISCUSSION: Our allergy reconciliation module achieved high performance, and physicians who used it largely accepted its recommendations. However, 56% of the pilot group ultimately did not use the module. User engagement and education are likely needed to increase adoption. CONCLUSION: We built a module to automatically identify discrepancies within patients' allergy records and remind providers to reconcile and update the allergy list. Its high accuracy shows promise for enhancing patient safety and utility of drug allergy alerts.

OBJECTIVE: To assess the appropriateness of medication-related clinical decision support (CDS) alerts associated with renal insufficiency and the potential/actual harm from overriding the alerts. MATERIALS AND METHODS: Override rate frequency was recorded for all inpatients who had a renal CDS alert trigger between 05/2017 and 04/2018. Two random samples of 300 for each of 2 types of medication-related CDS alerts associated with renal insufficiency-"dose change" and "avoid medication"-were evaluated by 2 independent reviewers using predetermined criteria for appropriateness of alert trigger, appropriateness of override, and patient harm. RESULTS: We identified 37 100 "dose change" and 5095 "avoid medication" alerts in the population evaluated, and 100% of each were overridden. Dose change triggers were classified as 12.5% appropriate and overrides of these alerts classified as 90.5% appropriate. Avoid medication triggers were classified as 29.6% appropriate and overrides 76.5% appropriate. We identified 5 adverse drug events, and, of these, 4 of the 5 were due to inappropriately overridden alerts. CONCLUSION: Alerts were nearly always presented inappropriately and were all overridden during the 1-year period studied. Alert fatigue resulting from receiving too many poor-quality alerts may result in failure to recognize errors that could lead to patient harm. Although medication-related CDS alerts associated with renal insufficiency had previously been found to be the most clinically beneficial alerts in a legacy system, in this system they were ineffective. These findings underscore the need for improvements in alert design, implementation, and monitoring of alert performance to make alerts more patient-specific and clinically appropriate.

OBJECTIVE: The study sought to evaluate the overall performance of hospitals that used the Computerized Physician Order Entry Evaluation Tool in both 2017 and 2018, along with their performance against fatal orders and nuisance orders. MATERIALS AND METHODS: We evaluated 1599 hospitals that took the test in both 2017 and 2018 by using their overall percentage scores on the test, along with the percentage of fatal orders appropriately alerted on, and the percentage of nuisance orders incorrectly alerted on. RESULTS: Hospitals showed overall improvement; the mean score in 2017 was 58.1%, and this increased to 66.2% in 2018. Fatal order performance improved slightly from 78.8% to 83.0% (P < .001), though there was almost no change in nuisance order performance (89.0% to 89.7%; P = .43). Hospitals alerting on one or more nuisance orders had a 3-percentage-point increase in their overall score. DISCUSSION: Despite the improvement of overall scores in 2017 and 2018, there was little improvement in fatal order performance, suggesting that hospitals are not targeting the deadliest orders first. Nuisance order performance showed almost no improvement, and some hospitals may be achieving higher scores by overalerting, suggesting that the thresholds for which alerts are fired from are too low. CONCLUSIONS: Although hospitals improved overall from 2017 to 2018, there is still important room for improvement for both fatal and nuisance orders. Hospitals that incorrectly alerted on one or more nuisance orders had slightly higher overall performance, suggesting that some hospitals may be achieving higher scores at the cost of overalerting, which has the potential to cause clinician burnout and even worsen safety.

OBJECTIVE: The study sought to determine frequency and appropriateness of overrides of high-priority drug-drug interaction (DDI) alerts and whether adverse drug events (ADEs) were associated with overrides in a newly implemented electronic health record. MATERIALS AND METHODS: We conducted a retrospective study of overridden high-priority DDI alerts occurring from April 1, 2016, to March 31, 2017, from inpatient and outpatient settings at an academic health center. We studied highest-severity DDIs that were previously designated as "hard stops" and additional high-priority DDIs identified from clinical experience and literature review. All highest-severity alert overrides (n = 193) plus a stratified random sample of additional overrides (n = 371) were evaluated for override appropriateness, using predetermined criteria. Charts were reviewed to identify ADEs for overrides that resulted in medication administration. A chi-square test was used to compare ADE rate by override appropriateness. RESULTS: Of 16 011 alerts presented to providers, 15 318 (95.7%) were overridden, including 193 (87.3%) of the highest-severity DDIs and 15 125 (95.8%) of additional DDIs. Override appropriateness was 45.4% overall, 0.5% for highest-severity DDIs and 68.7% for additional DDIs. For alerts that resulted in medication administration (n = 423, 75.0%), 29 ADEs were identified (6.9%, 5.1 per 100 overrides). The rate of ADEs was higher with inappropriate vs appropriate overrides (9.4% vs 4.3%; P = .038). CONCLUSIONS: The override rate was nearly 90% for even the highest-severity DDI alerts, indicating that stronger suggestions should be made for these alerts, while other alerts should be evaluated for potential suppression.

OBJECTIVE: Incomplete and static reaction picklists in the allergy module led to free-text and missing entries that inhibit the clinical decision support intended to prevent adverse drug reactions. We developed a novel, data-driven, "dynamic" reaction picklist to improve allergy documentation in the electronic health record (EHR). MATERIALS AND METHODS: We split 3 decades of allergy entries in the EHR of a large Massachusetts healthcare system into development and validation datasets. We consolidated duplicate allergens and those with the same ingredients or allergen groups. We created a reaction value set via expert review of a previously developed value set and then applied natural language processing to reconcile reactions from structured and free-text entries. Three association rule-mining measures were used to develop a comprehensive reaction picklist dynamically ranked by allergen. The dynamic picklist was assessed using recall at top k suggested reactions, comparing performance to the static picklist. RESULTS: The modified reaction value set contained 490 reaction concepts. Among 4 234 327 allergy entries collected, 7463 unique consolidated allergens and 469 unique reactions were identified. Of the 3 dynamic reaction picklists developed, the 1 with the optimal ranking achieved recalls of 0.632, 0.763, and 0.822 at the top 5, 10, and 15, respectively, significantly outperforming the static reaction picklist ranked by reaction frequency. CONCLUSION: The dynamic reaction picklist developed using EHR data and a statistical measure was superior to the static picklist and suggested proper reactions for allergy documentation. Further studies might evaluate the usability and impact on allergy documentation in the EHR.

Objective: To develop a comprehensive value set for documenting and encoding adverse reactions in the allergy module of an electronic health record. Materials and Methods: We analyzed 2 471 004 adverse reactions stored in Partners Healthcare's Enterprise-wide Allergy Repository (PEAR) of 2.7 million patients. Using the Medical Text Extraction, Reasoning, and Mapping System, we processed both structured and free-text reaction entries and mapped them to Systematized Nomenclature of Medicine - Clinical Terms. We calculated the frequencies of reaction concepts, including rare, severe, and hypersensitivity reactions. We compared PEAR concepts to a Federal Health Information Modeling and Standards value set and University of Nebraska Medical Center data, and then created an integrated value set. Results: We identified 787 reaction concepts in PEAR. Frequently reported reactions included: rash (14.0%), hives (8.2%), gastrointestinal irritation (5.5%), itching (3.2%), and anaphylaxis (2.5%). We identified an additional 320 concepts from Federal Health Information Modeling and Standards and the University of Nebraska Medical Center to resolve gaps due to missing and partial matches when comparing these external resources to PEAR. This yielded 1106 concepts in our final integrated value set. The presence of rare, severe, and hypersensitivity reactions was limited in both external datasets. Hypersensitivity reactions represented roughly 20% of the reactions within our data. Discussion: We developed a value set for encoding adverse reactions using a large dataset from one health system, enriched by reactions from 2 large external resources. This integrated value set includes clinically important severe and hypersensitivity reactions. Conclusion: This work contributes a value set, harmonized with existing data, to improve the consistency and accuracy of reaction documentation in electronic health records, providing the necessary building blocks for more intelligent clinical decision support for allergies and adverse reactions.

Objective: To define the types and numbers of inpatient clinical decision support alerts, measure the frequency with which they are overridden, and describe providers' reasons for overriding them and the appropriateness of those reasons. Materials and Methods: We conducted a cross-sectional study of medication-related clinical decision support alerts over a 3-year period at a 793-bed tertiary-care teaching institution. We measured the rate of alert overrides, the rate of overrides by alert type, the reasons cited for overrides, and the appropriateness of those reasons. Results: Overall, 73.3% of patient allergy, drug-drug interaction, and duplicate drug alerts were overridden, though the rate of overrides varied by alert type (P < .0001). About 60% of overrides were appropriate, and that proportion also varied by alert type (P < .0001). Few overrides of renal- (2.2%) or age-based (26.4%) medication substitutions were appropriate, while most duplicate drug (98%), patient allergy (96.5%), and formulary substitution (82.5%) alerts were appropriate. Discussion: Despite warnings of potential significant harm, certain categories of alert overrides were inappropriate >75% of the time. The vast majority of duplicate drug, patient allergy, and formulary substitution alerts were appropriate, suggesting that these categories of alerts might be good targets for refinement to reduce alert fatigue. Conclusion: Almost three-quarters of alerts were overridden, and 40% of the overrides were not appropriate. Future research should optimize alert types and frequencies to increase their clinical relevance, reducing alert fatigue so that important alerts are not inappropriately overridden.

Objective: To estimate the national cost of ADEs resulting from inappropriate medication-related alert overrides in the U.S. inpatient setting. Materials and Methods: We used three different regression models (Basic, Model 1, Model 2) with model inputs taken from the medical literature. A random sample of 40 990 adult inpatients at the Brigham and Women's Hospital (BWH) in Boston with a total of 1 639 294 medication orders was taken. We extrapolated BWH medication orders using 2014 National Inpatient Sample (NIS) data. Results: Using three regression models, we estimated that 29.7 million adult inpatient discharges in 2014 resulted in between 1.02 billion and 1.07 billion medication orders, which in turn generated between 75.1 million and 78.8 million medication alerts, respectively. Taking the basic model (78.8 million), we estimated that 5.5 million medication-related alerts might have been inappropriately overridden, resulting in approximately 196 600 ADEs nationally. This was projected to cost between $871 million and $1.8 billion for treating preventable ADEs. We also estimated that clinicians and pharmacists would have jointly spent 175 000 hours responding to 78.8 million alerts with an opportunity cost of $16.9 million. Discussion and Conclusion: These data suggest that further optimization of hospitals computerized provider order entry systems and their associated clinical decision support is needed and would result in substantial savings. We have erred on the side of caution in developing this range, taking two conservative cost estimates for a preventable ADE that did not include malpractice or litigation costs, or costs of injuries to patients.

OBJECTIVE: Variation in the use of tests and treatments has been demonstrated to be substantial between providers and geographic regions. This study assessed variation between outpatient providers in overriding electronic prescribing warnings. METHODS: Responses to warnings were prospectively logged. Random effects models were used to calculate provider-to-provider variation in the rates for the decisions to override warnings in 6 different clinical domains: medication allergies, drug-drug interactions, duplicate drugs, renal recommendations, age-based recommendations, and formulary substitutions. RESULTS: A total of 157 482 responses were logged. Differences between 1717 providers accounted for 11% of the overall variability in override rates, so that while the average override rate was 45.2%, individual provider rates had a wide range with a 95% confidence interval (CI) (13.7%-76.7% ). The highest variations between providers were observed in the categories age-based (25.4% of total variability; average override rate 70.2% [95% CI, 29.1%-100% ]) and renal recommendations (24.2%; average 70% [95% CI, 29.5%-100% ]), and provider responses within these 2 categories were most often clinically inappropriate according to prior work. Among providers who received at least 10 age-based recommendations, 64 of 238 (27%) overrode ≥ 90% of the warnings and 13 of 238 (5%) overrode all of them. Of those who received at least 10 renal recommendations, 36 of 92 (39%) overrode ≥ 90% of the alerts and 9 of 92 (10%) overrode all of them. CONCLUSIONS: The decision to override prescribing warnings shows variation between providers, and the magnitude of variation differs among the clinical domains of the warnings; more variation was observed in areas with more inappropriate overrides.

BACKGROUND: Experts suggest that formulary alerts at the time of medication order entry are the most effective form of clinical decision support to automate formulary management. OBJECTIVE: Our objectives were to quantify the frequency of inappropriate nonformulary medication (NFM) alert overrides in the inpatient setting and provide insight on how the design of formulary alerts could be improved. METHODS: Alert overrides of the top 11 (n = 206) most-utilized and highest-costing NFMs, from January 1 to December 31, 2012, were randomly selected for appropriateness evaluation. Using an empirically developed appropriateness algorithm, appropriateness of NFM alert overrides was assessed by 2 pharmacists via chart review. Appropriateness agreement of overrides was assessed with a Cohen's kappa. We also assessed which types of NFMs were most likely to be inappropriately overridden, the override reasons that were disproportionately provided in the inappropriate overrides, and the specific reasons the overrides were considered inappropriate. RESULTS: Approximately 17.2% (n = 35.4/206) of NFM alerts were inappropriately overridden. Non-oral NFM alerts were more likely to be inappropriately overridden compared to orals. Alerts overridden with "blank" reasons were more likely to be inappropriate. The failure to first try a formulary alternative was the most common reason for alerts being overridden inappropriately. CONCLUSION: Approximately 1 in 5 NFM alert overrides are overridden inappropriately. Future research should evaluate the impact of mandating a valid override reason and adding a list of formulary alternatives to each NFM alert; we speculate these NFM alert features may decrease the frequency of inappropriate overrides.

OBJECTIVE: Accurate food adverse sensitivity documentation in electronic health records (EHRs) is crucial to patient safety. This study examined, encoded, and grouped foods that caused any adverse sensitivity in a large allergy repository using natural language processing and standard terminologies. METHODS: Using the Medical Text Extraction, Reasoning, and Mapping System (MTERMS), we processed both structured and free-text entries stored in an enterprise-wide allergy repository (Partners' Enterprise-wide Allergy Repository), normalized diverse food allergen terms into concepts, and encoded these concepts using the Systematized Nomenclature of Medicine - Clinical Terms (SNOMED-CT) and Unique Ingredient Identifiers (UNII) terminologies. Concept coverage also was assessed for these two terminologies. We further categorized allergen concepts into groups and calculated the frequencies of these concepts by group. Finally, we conducted an external validation of MTERMS's performance when identifying food allergen terms, using a randomized sample from a different institution. RESULTS: We identified 158 552 food allergen records (2140 unique terms) in the Partners repository, corresponding to 672 food allergen concepts. High-frequency groups included shellfish (19.3%), fruits or vegetables (18.4%), dairy (9.0%), peanuts (8.5%), tree nuts (8.5%), eggs (6.0%), grains (5.1%), and additives (4.7%). Ambiguous, generic concepts such as "nuts" and "seafood" accounted for 8.8% of the records. SNOMED-CT covered more concepts than UNII in terms of exact (81.7% vs 68.0%) and partial (14.3% vs 9.7%) matches. DISCUSSION: Adverse sensitivities to food are diverse, and existing standard terminologies have gaps in their coverage of the breadth of allergy concepts. CONCLUSION: New strategies are needed to represent and standardize food adverse sensitivity concepts, to improve documentation in EHRs.

OBJECTIVE: There have been growing concerns about the impact of drug allergy alerts on patient safety and provider alert fatigue. The authors aimed to explore the common drug allergy alerts over the last 10 years and the reasons why providers tend to override these alerts. DESIGN: Retrospective observational cross-sectional study (2004-2013). MATERIALS AND METHODS: Drug allergy alert data (n = 611,192) were collected from two large academic hospitals in Boston, MA (USA). RESULTS: Overall, the authors found an increase in the rate of drug allergy alert overrides, from 83.3% in 2004 to 87.6% in 2013 (P < .001). Alarmingly, alerts for immune mediated and life threatening reactions with definite allergen and prescribed medication matches were overridden 72.8% and 74.1% of the time, respectively. However, providers were less likely to override these alerts compared to possible (cross-sensitivity) or probable (allergen group) matches (P < .001). The most common drug allergy alerts were triggered by allergies to narcotics (48%) and other analgesics (6%), antibiotics (10%), and statins (2%). Only slightly more than one-third of the reactions (34.2%) were potentially immune mediated. Finally, more than half of the overrides reasons pointed to irrelevant alerts (i.e., patient has tolerated the medication before, 50.9%) and providers were significantly more likely to override repeated alerts (89.7%) rather than first time alerts (77.4%, P < .001). DISCUSSION AND CONCLUSIONS: These findings underline the urgent need for more efforts to provide more accurate and relevant drug allergy alerts to help reduce alert override rates and improve alert fatigue.

BACKGROUND: Electronic prescribing is increasingly used, in part because of government incentives for its use. Many of its benefits come from clinical decision support (CDS), but often too many alerts are displayed, resulting in alert fatigue. OBJECTIVE: To characterize the override rates for medication-related CDS alerts in the outpatient setting, the reasons cited for overrides at the time of prescribing, and the appropriateness of overrides. METHODS: We measured CDS alert override rates and the coded reasons for overrides cited by providers at the time of prescribing. Our primary outcome was the rate of CDS alert overrides; our secondary outcomes were the rate of overrides by alert type, reasons cited for overrides at the time of prescribing, and override appropriateness for a subset of 600 alert overrides. Through detailed chart reviews of alert override cases, and selective literature review, we developed appropriateness criteria for each alert type, which were modified iteratively as necessary until consensus was reached on all criteria. RESULTS: We reviewed 157,483 CDS alerts (7.9% alert rate) on 2,004,069 medication orders during the study period. 82,889 (52.6%) of alerts were overridden. The most common alerts were duplicate drug (33.1%), patient allergy (16.8%), and drug-drug interactions (15.8%). The most likely alerts to be overridden were formulary substitutions (85.0%), age-based recommendations (79.0%), renal recommendations (78.0%), and patient allergies (77.4%). An average of 53% of overrides were classified as appropriate, and rates of appropriateness varied by alert type (p<0.0001) from 12% for renal recommendations to 92% for patient allergies. DISCUSSION: About half of CDS alerts were overridden by providers and about half of the overrides were classified as appropriate, but the likelihood of overriding an alert varied widely by alert type. Refinement of these alerts has the potential to improve the relevance of alerts and reduce alert fatigue.

BACKGROUND: Clinical decision support systems can prevent knowledge-based prescription errors and improve patient outcomes. The clinical effectiveness of these systems, however, is substantially limited by poor user acceptance of presented warnings. To enhance alert acceptance it may be useful to quantify the impact of potential modulators of acceptance. METHODS: We built a logistic regression model to predict alert acceptance of drug-drug interaction (DDI) alerts in three different settings. Ten variables from the clinical and human factors literature were evaluated as potential modulators of provider alert acceptance. ORs were calculated for the impact of knowledge quality, alert display, textual information, prioritization, setting, patient age, dose-dependent toxicity, alert frequency, alert level, and required acknowledgment on acceptance of the DDI alert. RESULTS: 50,788 DDI alerts were analyzed. Providers accepted only 1.4% of non-interruptive alerts. For interruptive alerts, user acceptance positively correlated with frequency of the alert (OR 1.30, 95% CI 1.23 to 1.38), quality of display (4.75, 3.87 to 5.84), and alert level (1.74, 1.63 to 1.86). Alert acceptance was higher in inpatients (2.63, 2.32 to 2.97) and for drugs with dose-dependent toxicity (1.13, 1.07 to 1.21). The textual information influenced the mode of reaction and providers were more likely to modify the prescription if the message contained detailed advice on how to manage the DDI. CONCLUSION: We evaluated potential modulators of alert acceptance by assessing content and human factors issues, and quantified the impact of a number of specific factors which influence alert acceptance. This information may help improve clinical decision support systems design.

The objective of this review is to describe the implementation of human factors principles for the design of alerts in clinical information systems. First, we conduct a review of alarm systems to identify human factors principles that are employed in the design and implementation of alerts. Second, we review the medical informatics literature to provide examples of the implementation of human factors principles in current clinical information systems using alerts to provide medication decision support. Last, we suggest actionable recommendations for delivering effective clinical decision support using alerts. A review of studies from the medical informatics literature suggests that many basic human factors principles are not followed, possibly contributing to the lack of acceptance of alerts in clinical information systems. We evaluate the limitations of current alerting philosophies and provide recommendations for improving acceptance of alerts by incorporating human factors principles in their design.

OBJECTIVE: Interruptive alerts within electronic applications can cause "alert fatigue" if they fire too frequently or are clinically reasonable only some of the time. We assessed the impact of non-interruptive, real-time medication laboratory alerts on provider lab test ordering. DESIGN: We enrolled 22 outpatient practices into a prospective, randomized, controlled trial. Clinics either used the existing system or received on-screen recommendations for baseline laboratory tests when prescribing new medications. Since the warnings were non-interruptive, providers did not have to act upon or acknowledge the notification to complete a medication request. MEASUREMENTS: Data were collected each time providers performed suggested laboratory testing within 14 days of a new prescription order. Findings were adjusted for patient and provider characteristics as well as patient clustering within clinics. RESULTS: Among 12 clinics with 191 providers in the control group and 10 clinics with 175 providers in the intervention group, there were 3673 total events where baseline lab tests would have been advised: 1988 events in the control group and 1685 in the intervention group. In the control group, baseline labs were requested for 771 (39%) of the medications. In the intervention group, baseline labs were ordered by clinicians in 689 (41%) of the cases. Overall, no significant association existed between the intervention and the rate of ordering appropriate baseline laboratory tests. CONCLUSION: We found that non-interruptive medication laboratory monitoring alerts were not effective in improving receipt of recommended baseline laboratory test monitoring for medications. Further work is necessary to optimize compliance with non-critical recommendations.

OBJECTIVE: Few data exist measuring the effect of differentiating drug-drug interaction (DDI) alerts in computerized provider order entry systems (CPOE) by level of severity ("tiering"). We sought to determine if rates of provider compliance with DDI alerts in the inpatient setting differed when a tiered presentation was implemented. DESIGN: We performed a retrospective analysis of alert log data on hospitalized patients at two academic medical centers during the period from 2/1/2004 through 2/1/2005. Both inpatient CPOE systems used the same DDI checking service, but one displayed alerts differentially by severity level (tiered presentation, including hard stops for the most severe alerts) while the other did not. Participants were adult inpatients who generated a DDI alert, and providers who wrote the orders. Alerts were presented during the order entry process, providing the clinician with the opportunity to change the patient's medication orders to avoid the interaction. MEASUREMENTS: Rate of compliance to alerts at a tiered site compared to a non-tiered site. RESULTS: We reviewed 71,350 alerts, of which 39,474 occurred at the non-tiered site and 31,876 at the tiered site. Compliance with DDI alerts was significantly higher at the site with tiered DDI alerts compared to the non-tiered site (29% vs. 10%, p < 0.001). At the tiered site, 100% of the most severe alerts were accepted, vs. only 34% at the non-tiered site; moderately severe alerts were also more likely to be accepted at the tiered site (29% vs. 10%). CONCLUSION: Tiered alerting by severity was associated with higher compliance rates of DDI alerts in the inpatient setting, and lack of tiering was associated with a high override rate of more severe alerts.

Computerized drug prescribing alerts can improve patient safety, but are often overridden because of poor specificity and alert overload. Our objective was to improve clinician acceptance of drug alerts by designing a selective set of drug alerts for the ambulatory care setting and minimizing workflow disruptions by designating only critical to high-severity alerts to be interruptive to clinician workflow. The alerts were presented to clinicians using computerized prescribing within an electronic medical record in 31 Boston-area practices. There were 18,115 drug alerts generated during our six-month study period. Of these, 12,933 (71%) were noninterruptive and 5,182 (29%) interruptive. Of the 5,182 interruptive alerts, 67% were accepted. Reasons for overrides varied for each drug alert category and provided potentially useful information for future alert improvement. These data suggest that it is possible to design computerized prescribing decision support with high rates of alert recommendation acceptance by clinicians.

Few data exist about the impact of computerized prescribing systems on outpatient medication errors (MEs) and adverse drug events (ADEs). We compared the rates of MEs and ADEs in handwritten sites versus sites with basic computerized prescribing. These systems reduced ME rates but did not significantly reduce ADE rates. Failure to monitor accounted for a large percentage of preventable ADEs. More advanced computerized prescribing systems with decision support and monitoring functions may be necessary to reduce outpatient ADE rates.

BACKGROUND: Medication errors are common, and while most such errors have little potential for harm they cause substantial extra work in hospitals. A small proportion do have the potential to cause injury, and some cause preventable adverse drug events. OBJECTIVE: To evaluate the impact of computerized physician order entry (POE) with decision support in reducing the number of medication errors. DESIGN: Prospective time series analysis, with four periods. SETTING AND PARTICIPANTS: All patients admitted to three medical units were studied for seven to ten-week periods in four different years. The baseline period was before implementation of POE, and the remaining three were after. Sophistication of POE increased with each successive period. INTERVENTION: Physician order entry with decision support features such as drug allergy and drug-drug interaction warnings. MAIN OUTCOME MEASURE: Medication errors, excluding missed dose errors. RESULTS: During the study, the non-missed-dose medication error rate fell 81 percent, from 142 per 1,000 patient-days in the baseline period to 26.6 per 1,000 patient-days in the final period (P < 0.0001). Non-intercepted serious medication errors (those with the potential to cause injury) fell 86 percent from baseline to period 3, the final period (P = 0.0003). Large differences were seen for all main types of medication errors: dose errors, frequency errors, route errors, substitution errors, and allergies. For example, in the baseline period there were ten allergy errors, but only two in the following three periods combined (P < 0.0001). CONCLUSIONS: Computerized POE substantially decreased the rate of non-missed-dose medication errors. A major reduction in errors was achieved with the initial version of the system, and further reductions were found with addition of decision support features.

BACKGROUND: Adverse drug events (ADEs) are both common and costly. Most hospitals identify ADEs using spontaneous reporting, but this approach lacks sensitivity; chart review identifies more events but is expensive. Computer-based approaches to ADE identification appear promising, but they have not been directly compared with chart review and they are not widely used. OBJECTIVES: To develop a computer-based ADE monitor, and to compare the rate and type of ADEs found with the monitor with those discovered by chart review and by stimulated voluntary report. DESIGN: Prospective cohort study in one tertiary-care hospital. PARTICIPANTS: All patients admitted to nine medical and surgical units in a tertiary-care hospital over an eight-month period. MAIN OUTCOME MEASURE: Adverse drug events identified by the computer-based monitor, by chart review, and by stimulated voluntary report. METHODS: A computer-based monitoring program identified alerts, which were situations suggesting that an ADE might be present (e.g., an order for an antidote such as naloxone). A trained reviewer then examined patients' hospital records to determine whether an ADE had occurred. The results of the computer-based monitoring strategy were compared with two other ADE detection strategies: intensive chart review and stimulated voluntary report by nurses and pharmacists. The monitor and the chart review strategies were independent, and the reviewers were blinded. RESULTS: The computer monitoring strategy identified 2,620 alerts, of which 275 were determined to be ADEs. The chart review found 398 ADEs, whereas voluntary report detected 23. Of the 617 ADEs detected by at least one method, 76 ADEs were detected by both computer monitor and chart review. The computer monitor identified 45 percent; chart review, 65 percent; and voluntary report, 4 percent. The ADEs identified by computer monitor were more likely to be classified as "severe" than were those identified by chart review (51 versus 42 percent, p = .04). The positive predictive value of computer-generated alerts was 16 percent during the first eight weeks of the study; rule modifications increased this to 23 percent in the final eight weeks. The computer strategy required 11 person-hours per week to execute, whereas chart review required 55 person-hours per week and voluntary report strategy required 5. CONCLUSIONS: The computer-based monitor identified fewer ADEs than did chart review but many more ADEs than did stimulated voluntary report. The overlap among the ADEs identified using different methods was small, suggesting that the incidence of ADEs may be higher than previously reported and that different detection methods capture different events. The computer-based monitoring system represents an efficient approach for measuring ADE frequency and gauging the effectiveness of ADE prevention programs.