BACKGROUND: Chatbots are increasingly used to deliver health education, patient engagement, and access to healthcare services. GARDE-Chat is an open-source platform designed to facilitate the development, deployment, and dissemination of chatbot-based digital health interventions across different domains and settings. MATERIALS AND METHODS: GARDE-Chat was developed through an iterative process informed by real-world use cases to guide prioritization of key features. The tool was developed as an open-source platform to promote collaboration, broad dissemination, and impact across research and clinical domains. RESULTS: GARDE-Chat's main features include (1) a visual authoring interface that allows non-programmers to design chatbots; (2) support for scripted, large language model (LLM)-based and hybrid chatbots; (3) capacity to share chatbots with researchers and institutions; (4) integration with external applications and data sources such as electronic health records and REDCap; (5) delivery via web browsers or text messaging; and (6) detailed audit log supporting analyses of chatbot user interactions. Since its first release in July 2022, GARDE-Chat has supported the development of chatbot-based interventions tested in multiple studies, including large pragmatic clinical trials addressing topics such as genetic testing, COVID-19 testing, tobacco cessation, and cancer screening. DISCUSSION: Ongoing challenges include the effort required for developing chatbot scripts, ensuring safe use of LLMs, and integrating with clinical systems. CONCLUSION: GARDE-Chat is a generalizable platform for creating, implementing, and disseminating scalable chatbot-based population health interventions. It has been validated in several studies, and it is available to researchers and healthcare systems through an open-source mechanism.

OBJECTIVES: Patient care using genetics presents complex challenges. Clinical decision support (CDS) tools are a potential solution because they provide patient-specific risk assessments and/or recommendations at the point of care. This systematic review evaluated the literature on CDS systems which have been implemented to support genetically guided precision medicine (GPM). MATERIALS AND METHODS: A comprehensive search was conducted in MEDLINE and Embase, encompassing January 1, 2011-March 14, 2023. The review included primary English peer-reviewed research articles studying humans, focused on the use of computers to guide clinical decision-making and delivering genetically guided, patient-specific assessments, and/or recommendations to healthcare providers and/or patients. RESULTS: The search yielded 3832 unique articles. After screening, 41 articles were identified that met the inclusion criteria. Alerts and reminders were the most common form of CDS used. About 27 systems were integrated with the electronic health record; 2 of those used standards-based approaches for genomic data transfer. Three studies used a framework to analyze the implementation strategy. DISCUSSION: Findings include limited use of standards-based approaches for genomic data transfer, system evaluations that do not employ formal frameworks, and inconsistencies in the methodologies used to assess genetic CDS systems and their impact on patient outcomes. CONCLUSION: We recommend that future research on CDS system implementation for genetically GPM should focus on implementing more CDS systems, utilization of standards-based approaches, user-centered design, exploration of alternative forms of CDS interventions, and use of formal frameworks to systematically evaluate genetic CDS systems and their effects on patient care.

OBJECTIVES: To enhance the Business Process Management (BPM)+ Healthcare language portfolio by incorporating knowledge types not previously covered and to improve the overall effectiveness and expressiveness of the suite to improve Clinical Knowledge Interoperability. METHODS: We used the BPM+ Health and Object Management Group (OMG) standards development methodology to develop new languages, following a gap analysis between existing BPM+ Health languages and clinical practice guideline knowledge types. Proposal requests were developed based on these requirements, and submission teams were formed to respond to them. The resulting proposals were submitted to OMG for ratification. RESULTS: The BPM+ Health family of languages, which initially consisted of the Business Process Model and Notation, Decision Model and Notation, and Case Model and Notation, was expanded by adding 5 new language standards through the OMG. These include Pedigree and Provenance Model and Notation for expressing epistemic knowledge, Knowledge Package Model and Notation for supporting packaging knowledge, Shared Data Model and Notation for expressing ontic knowledge, Party Model and Notation for representing entities and organizations, and Specification Common Elements, a language providing a standard abstract and reusable library that underpins the 4 new languages. DISCUSSION AND CONCLUSION: In this effort, we adopted a strategy of separation of concerns to promote a portfolio of domain-agnostic, independent, but integrated domain-specific languages for authoring medical knowledge. This strategy is a practical and effective approach to expressing complex medical knowledge. These new domain-specific languages offer various knowledge-type options for clinical knowledge authors to choose from without potentially adding unnecessary overhead or complexity.

OBJECTIVE: This study aimed to 1) investigate algorithm enhancements for identifying patients eligible for genetic testing of hereditary cancer syndromes using family history data from electronic health records (EHRs); and 2) assess their impact on relative differences across sex, race, ethnicity, and language preference. MATERIALS AND METHODS: The study used EHR data from a tertiary academic medical center. A baseline rule-base algorithm, relying on structured family history data (structured data; SD), was enhanced using a natural language processing (NLP) component and a relaxed criteria algorithm (partial match [PM]). The identification rates and differences were analyzed considering sex, race, ethnicity, and language preference. RESULTS: Among 120,007 patients aged 25-60, detection rate differences were found across all groups using the SD (all P < 0.001). Both enhancements increased identification rates; NLP led to a 1.9 % increase and the relaxed criteria algorithm (PM) led to an 18.5 % increase (both P < 0.001). Combining SD with NLP and PM yielded a 20.4 % increase (P < 0.001). Similar increases were observed within subgroups. Relative differences persisted across most categories for the enhanced algorithms, with disproportionately higher identification of patients who are White, Female, non-Hispanic, and whose preferred language is English. CONCLUSION: Algorithm enhancements increased identification rates for patients eligible for genetic testing of hereditary cancer syndromes, regardless of sex, race, ethnicity, and language preference. However, differences in identification rates persisted, emphasizing the need for additional strategies to reduce disparities such as addressing underlying biases in EHR family health information and selectively applying algorithm enhancements for disadvantaged populations. Systematic assessment of differences in algorithm performance across population subgroups should be incorporated into algorithm development processes.

OBJECTIVES: To design an interface to support communication of machine learning (ML)-based prognosis for patients with advanced solid tumors, incorporating oncologists' needs and feedback throughout design. MATERIALS AND METHODS: Using an interdisciplinary user-centered design approach, we performed 5 rounds of iterative design to refine an interface, involving expert review based on usability heuristics, input from a color-blind adult, and 13 individual semi-structured interviews with oncologists. Individual interviews included patient vignettes and a series of interfaces populated with representative patient data and predicted survival for each treatment decision point when a new line of therapy (LoT) was being considered. Ongoing feedback informed design decisions, and directed qualitative content analysis of interview transcripts was used to evaluate usability and identify enhancement requirements. RESULTS: Design processes resulted in an interface with 7 sections, each addressing user-focused questions, supporting oncologists to "tell a story" as they discuss prognosis during a clinical encounter. The iteratively enhanced interface both triggered and reflected design decisions relevant when attempting to communicate ML-based prognosis, and exposed misassumptions. Clinicians requested enhancements that emphasized interpretability over explainability. Qualitative findings confirmed that previously identified issues were resolved and clarified necessary enhancements (eg, use months not days) and concerns about usability and trust (eg, address LoT received elsewhere). Appropriate use should be in the context of a conversation with an oncologist. CONCLUSION: User-centered design, ongoing clinical input, and a visualization to communicate ML-related outcomes are important elements for designing any decision support tool enabled by artificial intelligence, particularly when communicating prognosis risk.

INTRODUCTION: The use and interoperability of clinical knowledge starts with the quality of the formalism utilized to express medical expertise. However, a crucial challenge is that existing formalisms are often suboptimal, lacking the fidelity to represent complex knowledge thoroughly and concisely. Often this leads to difficulties when seeking to unambiguously capture, share, and implement the knowledge for care improvement in clinical information systems used by providers and patients. OBJECTIVES: To provide a systematic method to address some of the complexities of knowledge composition and interoperability related to standards-based representational formalisms of medical knowledge. METHODS: Several cross-industry (Healthcare, Linguistics, System Engineering, Standards Development, and Knowledge Engineering) frameworks were synthesized into a proposed reference knowledge framework. The framework utilizes IEEE 42010, the MetaObject Facility, the Semantic Triangle, an Ontology Framework, and the Domain and Comprehensibility Appropriateness criteria. The steps taken were: 1) identify foundational cross-industry frameworks, 2) select architecture description method, 3) define life cycle viewpoints, 4) define representation and knowledge viewpoints, 5) define relationships between neighboring viewpoints, and 6) establish characteristic definitions of the relationships between components. System engineering principles applied included separation of concerns, cohesion, and loose coupling. RESULTS: A "Multilayer Metamodel for Representation and Knowledge" (M*R/K) reference framework was defined. It provides a standard vocabulary for organizing and articulating medical knowledge curation perspectives, concepts, and relationships across the artifacts created during the life cycle of language creation, authoring medical knowledge, and knowledge implementation in clinical information systems such as electronic health records (EHR). CONCLUSION: M*R/K provides a systematic means to address some of the complexities of knowledge composition and interoperability related to medical knowledge representations used in diverse standards. The framework may be used to guide the development, assessment, and coordinated use of knowledge representation formalisms. M*R/K could promote the alignment and aggregated use of distinct domain-specific languages in composite knowledge artifacts such as clinical practice guidelines (CPGs).

Population health management (PHM) is an important approach to promote wellness and deliver health care to targeted individuals who meet criteria for preventive measures or treatment. A critical component for any PHM program is a data analytics platform that can target those eligible individuals. OBJECTIVE: The aim of this study was to design and implement a scalable standards-based clinical decision support (CDS) approach to identify patient cohorts for PHM and maximize opportunities for multi-site dissemination. MATERIALS AND METHODS: An architecture was established to support bidirectional data exchanges between heterogeneous electronic health record (EHR) data sources, PHM systems, and CDS components. HL7 Fast Healthcare Interoperability Resources and CDS Hooks were used to facilitate interoperability and dissemination. The approach was validated by deploying the platform at multiple sites to identify patients who meet the criteria for genetic evaluation of familial cancer. RESULTS: The Genetic Cancer Risk Detector (GARDE) platform was created and is comprised of four components: (1) an open-source CDS Hooks server for computing patient eligibility for PHM cohorts, (2) an open-source Population Coordinator that processes GARDE requests and communicates results to a PHM system, (3) an EHR Patient Data Repository, and (4) EHR PHM Tools to manage patients and perform outreach functions. Site-specific deployments were performed on onsite virtual machines and cloud-based Amazon Web Services. DISCUSSION: GARDE's component architecture establishes generalizable standards-based methods for computing PHM cohorts. Replicating deployments using one of the established deployment methods requires minimal local customization. Most of the deployment effort was related to obtaining site-specific information technology governance approvals.

Dear Editor, We agree with Drs. Tarabichi and Thornton on the importance of smoking history documentation in the electronic health record (EHR) for determining lung cancer screening eligibility. Drs. Tarabichi and Thornton also highlight an underappreciated advantage of lung cancer risk prediction: risk-based screening thresholds are less vulnerable to sub-optimally collected pack-year information. Their findings align with other work we have done where we found that unreliable pack-year information leads to less misclassification when using risk-based selection compared to using a pack-year eligibility criterion for patient selection.1 The growing body of literature on this topic, including those by ourselves, Drs. Tarabichi and Thornton, and others, underscores the potential for leveraging EHR capabilities to optimize the identification of high-risk patients for lung cancer screening.1–5 This body of literature underscores the ubiquitous and consequential nature of EHR smoking data inaccuracies and the need for additional research to generalize findings and to identify potential solutions.

OBJECTIVE: The US Preventive Services Task Force (USPSTF) requires the estimation of lifetime pack-years to determine lung cancer screening eligibility. Leading electronic health record (EHR) vendors calculate pack-years using only the most recently recorded smoking data. The objective was to characterize EHR smoking data issues and to propose an approach to addressing these issues using longitudinal smoking data. MATERIALS AND METHODS: In this cross-sectional study, we evaluated 16 874 current or former smokers who met USPSTF age criteria for screening (50-80 years old), had no prior lung cancer diagnosis, and were seen in 2020 at an academic health system using the Epic® EHR. We described and quantified issues in the smoking data. We then estimated how many additional potentially eligible patients could be identified using longitudinal data. The approach was verified through manual review of records from 100 subjects. RESULTS: Over 80% of evaluated records had inaccuracies, including missing packs-per-day or years-smoked (42.7%), outdated data (25.1%), missing years-quit (17.4%), and a recent change in packs-per-day resulting in inaccurate lifetime pack-years estimation (16.9%). Addressing these issues by using longitudinal data enabled the identification of 49.4% more patients potentially eligible for lung cancer screening (P < .001). DISCUSSION: Missing, outdated, and inaccurate smoking data in the EHR are important barriers to effective lung cancer screening. Data collection and analysis strategies that reflect changes in smoking habits over time could improve the identification of patients eligible for screening. CONCLUSION: The use of longitudinal EHR smoking data could improve lung cancer screening.

OBJECTIVE: To evaluate the potential for machine learning to predict medication alerts that might be ignored by a user, and intelligently filter out those alerts from the user's view. MATERIALS AND METHODS: We identified features (eg, patient and provider characteristics) proposed to modulate user responses to medication alerts through the literature; these features were then refined through expert review. Models were developed using rule-based and machine learning techniques (logistic regression, random forest, support vector machine, neural network, and LightGBM). We collected log data on alerts shown to users throughout 2019 at University of Utah Health. We sought to maximize precision while maintaining a false-negative rate <0.01, a threshold predefined through discussion with physicians and pharmacists. We developed models while maintaining a sensitivity of 0.99. Two null hypotheses were developed: H1-there is no difference in precision among prediction models; and H2-the removal of any feature category does not change precision. RESULTS: A total of 3,481,634 medication alerts with 751 features were evaluated. With sensitivity fixed at 0.99, LightGBM achieved the highest precision of 0.192 and less than 0.01 for the pre-defined maximal false-negative rate by subject-matter experts (H1) (P < 0.001). This model could reduce alert volume by 54.1%. We removed different combinations of features (H2) and found that not all features significantly contributed to precision. Removing medication order features (eg, dosage) most significantly decreased precision (-0.147, P = 0.001). CONCLUSIONS: Machine learning potentially enables the intelligent filtering of medication alerts.

OBJECTIVE: HL7 SMART on FHIR apps have the potential to improve healthcare delivery and EHR usability, but providers must be aware of the apps and use them for these potential benefits to be realized. The HL7 CDS Hooks standard was developed in part for this purpose. The objective of this study was to determine if contextually relevant CDS Hooks prompts can increase utilization of a SMART on FHIR medical reference app (MDCalc for EHR). MATERIALS AND METHODS: We conducted a 7-month, provider-randomized trial with 70 providers in a single emergency department. The intervention was a collection of CDS Hooks prompts suggesting the use of 6 medical calculators in a SMART on FHIR medical reference app. The primary outcome was the percentage of provider-patient interactions in which the app was used to view a recommended calculator. Secondary outcomes were app usage stratified by individual calculators. RESULTS: Intervention group providers viewed a study calculator in the app in 6.0% of interactions compared to 2.6% in the control group (odds ratio = 2.45, 95% CI, 1.2-5.2, P value .02), an increase of 130%. App use was significantly greater for 2 of 6 calculators. DISCUSSION AND CONCLUSION: Contextually relevant CDS Hooks prompts led to a significant increase in SMART on FHIR app utilization. This demonstrates the potential of using CDS Hooks to guide appropriate use of SMART on FHIR apps and was a primary motivation for the development of the standard. Future research may evaluate potential impacts on clinical care decisions and outcomes.

OBJECTIVE: The Genomic Medicine Working Group of the National Advisory Council for Human Genome Research virtually hosted its 13th genomic medicine meeting titled "Developing a Clinical Genomic Informatics Research Agenda". The meeting's goal was to articulate a research strategy to develop Genomics-based Clinical Informatics Tools and Resources (GCIT) to improve the detection, treatment, and reporting of genetic disorders in clinical settings. MATERIALS AND METHODS: Experts from government agencies, the private sector, and academia in genomic medicine and clinical informatics were invited to address the meeting's goals. Invitees were also asked to complete a survey to assess important considerations needed to develop a genomic-based clinical informatics research strategy. RESULTS: Outcomes from the meeting included identifying short-term research needs, such as designing and implementing standards-based interfaces between laboratory information systems and electronic health records, as well as long-term projects, such as identifying and addressing barriers related to the establishment and implementation of genomic data exchange systems that, in turn, the research community could help address. DISCUSSION: Discussions centered on identifying gaps and barriers that impede the use of GCIT in genomic medicine. Emergent themes from the meeting included developing an implementation science framework, defining a value proposition for all stakeholders, fostering engagement with patients and partners to develop applications under patient control, promoting the use of relevant clinical workflows in research, and lowering related barriers to regulatory processes. Another key theme was recognizing pervasive biases in data and information systems, algorithms, access, value, and knowledge repositories and identifying ways to resolve them.

OBJECTIVE: Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation. METHODS: The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies. RESULTS: The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment. DISCUSSION: Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation. CONCLUSION: As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

Electronic health record (EHR) data are increasingly used to develop prediction models to support clinical care, including the care of patients with common chronic conditions. A key challenge for individual healthcare systems in developing such models is that they may not be able to achieve the desired degree of robustness using only their own data. A potential solution-combining data from multiple sources-faces barriers such as the need for data normalization and concerns about sharing patient information across institutions. To address these challenges, we evaluated three alternative approaches to using EHR data from multiple healthcare systems in predicting the outcome of pharmacotherapy for type 2 diabetes mellitus(T2DM). Two of the three approaches, named Selecting Better (SB) and Weighted Average(WA), allowed the data to remain within institutional boundaries by using pre-built prediction models; the third, named Combining Data (CD), aggregated raw patient data into a single dataset. The prediction performance and prediction coverage of the resulting models were compared to single-institution models to help judge the relative value of adding external data and to determine the best method to generate optimal models for clinical decision support. The results showed that models using WA and CD achieved higher prediction performance than single-institution models for common treatment patterns. CD outperformed the other two approaches in prediction coverage, which we defined as the number of treatment patterns predicted with an Area Under Curve of 0.70 or more. We concluded that 1) WA is an effective option for improving prediction performance for common treatment patterns when data cannot be shared across institutional boundaries and 2) CD is the most effective approach when such sharing is possible, especially for increasing the range of treatment patterns that can be predicted to support clinical decision making.

OBJECTIVE: The purpose of the study was to explore the theoretical underpinnings of effective clinical decision support (CDS) factors using the comparative effectiveness results. MATERIALS AND METHODS: We leveraged search results from a previous systematic literature review and updated the search to screen articles published from January 2017 to January 2020. We included randomized controlled trials and cluster randomized controlled trials that compared a CDS intervention with and without specific factors. We used random effects meta-regression procedures to analyze clinician behavior for the aggregate effects. The theoretical model was the Unified Theory of Acceptance and Use of Technology (UTAUT) model with motivational control. RESULTS: Thirty-four studies were included. The meta-regression models identified the importance of effort expectancy (estimated coefficient = -0.162; P = .0003); facilitating conditions (estimated coefficient = 0.094; P = .013); and performance expectancy with motivational control (estimated coefficient = 1.029; P = .022). Each of these factors created a significant impact on clinician behavior. The meta-regression model with the multivariate analysis explained a large amount of the heterogeneity across studies (R2 = 88.32%). DISCUSSION: Three positive factors were identified: low effort to use, low controllability, and providing more infrastructure and implementation strategies to support the CDS. The multivariate analysis suggests that passive CDS could be effective if users believe the CDS is useful and/or social expectations to use the CDS intervention exist. CONCLUSIONS: Overall, a modified UTAUT model that includes motivational control is an appropriate model to understand psychological factors associated with CDS effectiveness and to guide CDS design, implementation, and optimization.

OBJECTIVE: To facilitate the development of standards-based clinical decision support (CDS) systems, we review the current set of CDS standards that are based on Health Level Seven International Fast Healthcare Interoperability Resources (FHIR). Widespread adoption of these standards may help reduce healthcare variability, improve healthcare quality, and improve patient safety. TARGET AUDIENCE: This tutorial is designed for the broad informatics community, some of whom may be unfamiliar with the current, FHIR-based CDS standards. SCOPE: This tutorial covers the following standards: Arden Syntax (using FHIR as the data model), Clinical Quality Language, FHIR Clinical Reasoning, SMART on FHIR, and CDS Hooks. Detailed descriptions and selected examples are provided.

OBJECTIVE: The study sought to evaluate a novel electronic health record (EHR) add-on application for chronic disease management that uses an integrated display to decrease user cognitive load, improve efficiency, and support clinical decision making. MATERIALS AND METHODS: We designed a chronic disease management application using the technology framework known as SMART on FHIR (Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources). We used mixed methods to obtain user feedback on a prototype to support ambulatory providers managing chronic obstructive pulmonary disease. Each participant managed 2 patient scenarios using the regular EHR with and without access to our prototype in block-randomized order. The primary outcome was the percentage of expert-recommended ideal care tasks completed. Timing, keyboard and mouse use, and participant surveys were also collected. User experiences were captured using a retrospective think-aloud interview analyzed by concept coding. RESULTS: With our prototype, the 13 participants completed more recommended care (81% vs 48%; P < .001) and recommended tasks per minute (0.8 vs 0.6; P = .03) over longer sessions (7.0 minutes vs 5.4 minutes; P = .006). Keystrokes per task were lower with the prototype (6 vs 18; P < .001). Qualitative themes elicited included the desire for reliable presentation of information which matches participants' mental models of disease and for intuitive navigation in order to decrease cognitive load. DISCUSSION: Participants completed more recommended care by taking more time when using our prototype. Interviews identified a tension between using the inefficient but familiar EHR vs learning to use our novel prototype. Concept coding of user feedback generated actionable insights. CONCLUSIONS: Mixed methods can support the design and evaluation of SMART on FHIR EHR add-on applications by enhancing understanding of the user experience.

OBJECTIVE: To determine the impact of a graphical information display on diagnosing circulatory shock. MATERIALS AND METHODS: This was an experimental study comparing integrated and conventional information displays. Participants were intensivists or critical care fellows (experts) and first-year medical residents (novices). RESULTS: The integrated display was associated with higher performance (87% vs 82%; P < .001), less time (2.9 vs 3.5 min; P = .008), and more accurate etiology (67% vs 54%; P = .048) compared to the conventional display. When stratified by experience, novice physicians using the integrated display had higher performance (86% vs 69%; P < .001), less time (2.9 vs 3.7 min; P = .03), and more accurate etiology (65% vs 42%; P = .02); expert physicians using the integrated display had nonsignificantly improved performance (87% vs 82%; P = .09), time (2.9 vs 3.3; P = .28), and etiology (69% vs 67%; P = .81). DISCUSSION: The integrated display appeared to support efficient information processing, which resulted in more rapid and accurate circulatory shock diagnosis. Evidence more strongly supported a difference for novices, suggesting that graphical displays may help reduce expert-novice performance gaps.

OBJECTIVE: The study sought to describe key features of clinical concepts and data required to implement clinical practice recommendations as clinical decision support (CDS) tools in electronic health record systems and to identify recommendation features that predict feasibility of implementation. MATERIALS AND METHODS: Using semistructured interviews, CDS implementers and clinician subject matter experts from 7 academic medical centers rated the feasibility of implementing 10 American College of Emergency Physicians Choosing Wisely Recommendations as electronic health record-embedded CDS and estimated the need for additional data collection. Ratings were combined with objective features of the guidelines to develop a predictive model for technical implementation feasibility. RESULTS: A linear mixed model showed that the need for new data collection was predictive of lower implementation feasibility. The number of clinical concepts in each recommendation, need for historical data, and ambiguity of clinical concepts were not predictive of implementation feasibility. CONCLUSIONS: The availability of data and need for additional data collection are essential to assess the feasibility of CDS implementation. Authors of practice recommendations and guidelines can enable organizations to more rapidly assess data availability and feasibility of implementation by including operational definitions for required data.

OBJECTIVE: To develop expeditiously a pragmatic, modular, and extensible software framework for understanding and improving healthcare value (costs relative to outcomes). MATERIALS AND METHODS: In 2012, a multidisciplinary team was assembled by the leadership of the University of Utah Health Sciences Center and charged with rapidly developing a pragmatic and actionable analytics framework for understanding and enhancing healthcare value. Based on an analysis of relevant prior work, a value analytics framework known as Value Driven Outcomes (VDO) was developed using an agile methodology. Evaluation consisted of measurement against project objectives, including implementation timeliness, system performance, completeness, accuracy, extensibility, adoption, satisfaction, and the ability to support value improvement. RESULTS: A modular, extensible framework was developed to allocate clinical care costs to individual patient encounters. For example, labor costs in a hospital unit are allocated to patients based on the hours they spent in the unit; actual medication acquisition costs are allocated to patients based on utilization; and radiology costs are allocated based on the minutes required for study performance. Relevant process and outcome measures are also available. A visualization layer facilitates the identification of value improvement opportunities, such as high-volume, high-cost case types with high variability in costs across providers. Initial implementation was completed within 6 months, and all project objectives were fulfilled. The framework has been improved iteratively and is now a foundational tool for delivering high-value care. CONCLUSIONS: The framework described can be expeditiously implemented to provide a pragmatic, modular, and extensible approach to understanding and improving healthcare value.

OBJECTIVE: To identify key principles for establishing a national clinical decision support (CDS) knowledge sharing framework. MATERIALS AND METHODS: As part of an initiative by the US Office of the National Coordinator for Health IT (ONC) to establish a framework for national CDS knowledge sharing, key stakeholders were identified. Stakeholders' viewpoints were obtained through surveys and in-depth interviews, and findings and relevant insights were summarized. Based on these insights, key principles were formulated for establishing a national CDS knowledge sharing framework. RESULTS: Nineteen key stakeholders were recruited, including six executives from electronic health record system vendors, seven executives from knowledge content producers, three executives from healthcare provider organizations, and three additional experts in clinical informatics. Based on these stakeholders' insights, five key principles were identified for effectively sharing CDS knowledge nationally. These principles are (1) prioritize and support the creation and maintenance of a national CDS knowledge sharing framework; (2) facilitate the development of high-value content and tooling, preferably in an open-source manner; (3) accelerate the development or licensing of required, pragmatic standards; (4) acknowledge and address medicolegal liability concerns; and (5) establish a self-sustaining business model. DISCUSSION: Based on the principles identified, a roadmap for national CDS knowledge sharing was developed through the ONC's Advancing CDS initiative. CONCLUSION: The study findings may serve as a useful guide for ongoing activities by the ONC and others to establish a national framework for sharing CDS knowledge and improving clinical care.

OBJECTIVE: To review the literature on clinical decision support (CDS) for genetically guided personalized medicine (GPM). MATERIALS AND METHODS: MEDLINE and Embase were searched from 1990 to 2011. The manuscripts included were summarized, and notable themes and trends were identified. RESULTS: Following a screening of 3416 articles, 38 primary research articles were identified. Focal areas of research included family history-driven CDS, cancer management, and pharmacogenomics. Nine randomized controlled trials of CDS interventions for GPM were identified, seven of which reported positive results. The majority of manuscripts were published on or after 2007, with increased recent focus on genotype-driven CDS and the integration of CDS within primary clinical information systems. DISCUSSION: Substantial research has been conducted to date on the use of CDS to enable GPM. In a previous analysis of CDS intervention trials, the automatic provision of CDS as a part of routine clinical workflow had been identified as being critical for CDS effectiveness. There was some indication that CDS for GPM could potentially be effective without the CDS being provided automatically, but we did not find conclusive evidence to support this hypothesis. CONCLUSION: To maximize the clinical benefits arising from ongoing discoveries in genetics and genomics, additional research and development is recommended for identifying how best to leverage CDS to bridge the gap between the promise and realization of GPM.

CONTEXT: The healthcare industry could achieve significant benefits through the adoption of a service-oriented architecture (SOA). The specification and adoption of standard software service interfaces will be critical to achieving these benefits. OBJECTIVE: To develop a replicable, collaborative framework for standardizing the interfaces of software services important to healthcare. DESIGN: Iterative, peer-reviewed development of a framework for generating interoperable service specifications that build on existing and ongoing standardization efforts. The framework was created under the auspices of the Healthcare Services Specification Project (HSSP), which was initiated in 2005 as a joint initiative between Health Level7 (HL7) and the Object Management Group (OMG). In this framework, known as the HSSP Service Specification Framework, HL7 identifies candidates for service standardization and defines normative Service Functional Models (SFMs) that specify the capabilities and conformance criteria for these services. OMG then uses these SFMs to generate technical service specifications as well as reference implementations. MEASUREMENTS: The ability of the framework to support the creation of multiple, interoperable service specifications useful for healthcare. RESULTS: Functional specifications have been defined through HL7 for four services: the Decision Support Service; the Entity Identification Service; the Clinical Research Filtered Query Service; and the Retrieve, Locate, and Update Service. Technical specifications and commercial implementations have been developed for two of these services within OMG. Furthermore, three additional functional specifications are being developed through HL7. CONCLUSIONS: The HSSP Service Specification Framework provides a replicable and collaborative approach to defining standardized service specifications for healthcare.

Despite their demonstrated effectiveness, clinical decision support (CDS) systems are not widely used within the U.S. The Roadmap for National Action on Clinical Decision Support, published in June 2006 by the American Medical Informatics Association, identifies six strategic objectives for achieving widespread adoption of effective CDS capabilities. In this manuscript, we propose a Service-Oriented Architecture (SOA) for CDS that facilitates achievement of these six objectives. Within the proposed framework, CDS capabilities are implemented through the orchestration of independent software services whose interfaces are being standardized by Health Level 7 and the Object Management Group through their joint Healthcare Services Specification Project (HSSP). Core services within this framework include the HSSP Decision Support Service, the HSSP Common Terminology Service, and the HSSP Retrieve, Locate, and Update Service. Our experiences, and those of others, indicate that the proposed SOA approach to CDS could enable the widespread adoption of effective CDS within the U.S. health care system.

Health care provides many opportunities in which the sharing of data between independent sites is highly desirable. Several standards are required to produce the functional and semantic interoperability necessary to support the exchange of such data: a common reference information model, a common set of data elements, a common terminology, common data structures, and a common transport standard. This paper addresses one component of that set of standards: the ability to create a document that supports the exchange of structured data components. Unfortunately, two different standards development organizations have produced similar standards for that purpose based on different information models: Health Level 7 (HL7)'s Clinical Document Architecture (CDA) and The American Society for Testing and Materials (ASTM International) Continuity of Care Record (CCR). The coexistence of both standards might require mapping from one standard to the other, which could be accompanied by a loss of information and functionality. This paper examines and compares the two standards, emphasizes the strengths and weaknesses of each, and proposes a strategy of harmonization to enhance future progress. While some of the authors are members of HL7 and/or ASTM International, the authors stress that the viewpoints represented in this paper are those of the authors and do not represent the official viewpoints of either HL7 or of ASTM International.