Computable biomedical knowledge (CBK) is: "the result of an analytic and/or deliberative process about human health, or affecting human health, that is explicit, and therefore can be represented and reasned upon using logic, formal standards, and mathematical approaches." Representing biomedical knowledge in a machine-interpretable, computable form increases its ability to be discovered, accessed, understood, and deployed. Computable knowledge artifacts can greatly advance the potential for implementation, reproducibility, or extension of the knowledge by users, who may include practitioners, researchers, and learners. Enriching computable knowledge artifacts may help facilitate reuse and translation into practice. Following the examples of 10 Simple Rules papers for scientific code, software, and applications, we present 10 Simple Rules intended to make shared computable knowledge artifacts more useful and reusable. These rules are mainly for researchers and their teams who have decided that sharing their computable knowledge is important, who wish to go beyond simply describing results, algorithms, or models via traditional publication pathways, and who want to both make their research findings more accessible, and to help others use their computable knowledge. These rules are roughly organized into 3 categories: planning, engineering, and documentation. Finally, while many of the following examples are of computable knowledge in biomedical domains, these rules are generalizable to computable knowledge in any research domain.

Dear Editors, We recognize that inefficient and idiosyncratic workflows in health care contribute to a myriad of obstacles for all healthcare stakeholders, including misuse of resources, provider burnout, and increased burden on patients and their caregivers.1 Therefore, we were pleased to read Zayas-Cabán et al’s2 article, “Priorities to accelerate workflow automation in healthcare.” We applaud them for their work illuminating determinants, priorities, and associated strategies for workflow automation. We augment their findings by proposing a seventh priority to stand alongside their original 6—develop and promote infrastructure to facilitate findability, accessibility, interoperability, and reusability of workflows (Table 1). In our view, the automation of healthcare workflows depends on the deployment of reliable, valid, robust, and proven computable biomedical knowledge (CBK) artifacts. We define CBK artifacts as separately packaged software implementations of evidence-based procedural, logical, mathematical, and statistical algorithms.3 We hold this view at an equal level of importance as the need for high-quality, interoperable data and a deep understanding of workflows, as emphasized by Zayas-Cabán et al. Our perspective is that automation of workflows will depend on formalizing and explicitly representing current and desired (optimal) workflows so that they can be tracked and processed by computers in valuable ways. Moreover, to achieve powerful automation, infrastructure must be developed to coordinate and combine computable workflows or process models with AI models, computable guidelines, and other CBK artifacts.

OBJECTIVE: To determine the extent of implementation, completeness, and accuracy of Structured and Codified SIG (S&C SIG) directions on electronic prescriptions (e-prescriptions). MATERIALS AND METHODS: A retrospective analysis of a random sample of 3.8 million e-prescriptions sent from electronic prescribing (e-prescribing) software to outpatient pharmacies in the United States between 2019 and 2021. Natural language processing was used to identify direction components, including action verb, dose, frequency, route, duration, and indication from free-text directions and were compared to the S&C SIG format. Inductive qualitative analysis of S&C direction identified error types and frequencies for each component. RESULTS: Implementation of the S&C SIG format in e-prescribing software resulted in 32.4% of e-prescriptions transmitted with these standardized directions. Directions using the S&C SIG format contained a greater percentage of each direction component compared to free-text directions, except for the indication component. Structured and codified directions contained quality issues in 10.3% of cases. DISCUSSION: Expanding adoption of more diverse direction terminology for the S&C SIG formats can improve the coverage of directions using the S&C SIG format. Building out e-prescribing software interfaces to include more direction components can improve patient medication use and safety. Quality improvement efforts, such as improving the design of e-prescribing software and auditing for discrepancies, are needed to identify and eliminate implementation-related issues with direction information from the S&C SIG format so that e-prescription directions are always accurately represented. CONCLUSION: Although directions using the S&C SIG format may result in more complete directions, greater adoption of the format and best practices for preventing its incorrect use are necessary.

OBJECTIVE: To determine the variability of ingredient, strength, and dose form information from drug product descriptions in real-world electronic prescription (e-prescription) data. MATERIALS AND METHODS: A sample of 10 399 324 e-prescriptions from 2019 to 2021 were obtained. Drug product descriptions were analyzed with a named entity extraction model and National Drug Codes (NDCs) were used to get RxNorm Concept Unique Identifiers (RxCUI) via RxNorm. The number of drug product description variants for each RxCUI was determined. Variants identified were compared to RxNorm to determine the extent of matching terminology used. RESULTS: A total of 353 002 unique pairs of drug product descriptions and NDCs were analyzed. The median (1st-3rd quartile) number of variants extracted for each standardized expression in RxNorm, was 3 (2-7) for ingredients, 4 (2-8) for strength, and 41 (11-122) for dosage forms. Of the pairs, 42.35% of ingredients (n = 328 032), 51.23% of strengths (n = 321 706), and 10.60% of dose forms (n = 326 653) used matching terminology, while 16.31%, 24.85%, and 13.05% contained nonmatching terminology, respectively. DISCUSSION: A wide variety of drug product descriptions makes it difficult to determine whether 2 drug product descriptions describe the same drug product (eg, using abbreviations to describe an active ingredient or using different units to represent a concentration). This results in patient safety risks that lead to incorrect drug products being ordered, dispensed, and used by patients. Implementation and use of standardized terminology may reduce these risks. CONCLUSION: Drug product descriptions on real-world e-prescriptions exhibit large variation resulting in unnecessary ambiguity and potential patient safety risks.

OBJECTIVES: The study sought to learn if it were possible to develop an ontology that would allow the Food and Drug Administration approved indications to be expressed in a manner computable and comparable to what is expressed in an electronic health record. MATERIALS AND METHODS: A random sample of 1177 of the 3000+ extant, distinct medical products (identified by unique new drug application numbers) was selected for investigation. Close manual examination of the indication portion of the labels for these drugs led to the development of a formal model of indications. RESULTS: The model represents each narrative indication as a disjunct of conjuncts of assertions about an individual. A desirable attribute is that each assertion about an individual should be testable without reference to other contextual information about the situation. The logical primitives are chosen from 2 categories (context and conditions) and are linked to an enumeration of uses, such as prevention. We found that more than 99% of approved label indications for treatment or prevention could be so represented. DISCUSSION: While some indications are straightforward to represent, difficulties stem from the need to represent temporal or sequential references. In addition, there is a mismatch of terminologies between what is present in an electronic health record and in the label narrative. CONCLUSIONS: A workable model for formalizing drug indications is possible. Remaining challenges include designing workflow to model narrative label indications for all approved drug products and incorporation of standard vocabularies.

The specificity of medication-related alerts must be improved to overcome the pernicious effects of alert fatigue. A systematic comparison of new drug orders to historical orders could improve alert specificity and relevance. Using historical order data from a computerized provider order entry system, we alerted physicians to atypical orders during the prescribing of five medications: calcium, clopidogrel, heparin, magnesium, and potassium. The percentage of atypical orders placed for these five medications decreased during the 92 days the alerts were active when compared to the same period in the previous year (from 0.81% to 0.53%; p=0.015). Some atypical orders were appropriate. Fifty of the 68 atypical order alerts were over-ridden (74%). However, the over-ride rate is misleading because 28 of the atypical medication orders (41%) were changed. Atypical order alerts were relatively few, identified problems with frequencies as well as doses, and had a higher specificity than dose check alerts.