Abstract There is an explosion in the number of ontologies and semantic artefacts being produced in science. This paper discusses the need for common platforms to receive, host, serve, align, and enable their reuse. Ontology repositories and semantic artefact catalogues are necessary to address this need and to make ontologies FAIR (Findable, Accessible, Interoperable, and Reusable). The OntoPortal Alliance ( https://ontoportal.org ) is a consortium of research and infrastructure teams dedicated to promoting the development of such repositories based on the open, collaboratively developed OntoPortal software. We present the OntoPortal technology as a generic resource to build ontology repositories and semantic artefact catalogues that can support resources ranging from SKOS thesauri to OBO, RDF-S, and OWL ontologies. The paper reviews the features of OntoPortal and presents the current and forthcoming public and open repositories built with the technology maintained by the Alliance.

MOTIVATION: Knowledge graphs (KGs) are being adopted in industry, commerce and academia. Biomedical KG presents a challenge due to the complexity, size and heterogeneity of the underlying information. RESULTS: In this work, we present the Scalable Precision Medicine Open Knowledge Engine (SPOKE), a biomedical KG connecting millions of concepts via semantically meaningful relationships. SPOKE contains 27 million nodes of 21 different types and 53 million edges of 55 types downloaded from 41 databases. The graph is built on the framework of 11 ontologies that maintain its structure, enable mappings and facilitate navigation. SPOKE is built weekly by python scripts which download each resource, check for integrity and completeness, and then create a 'parent table' of nodes and edges. Graph queries are translated by a REST API and users can submit searches directly via an API or a graphical user interface. Conclusions/Significance: SPOKE enables the integration of seemingly disparate information to support precision medicine efforts. AVAILABILITY AND IMPLEMENTATION: The SPOKE neighborhood explorer is available at https://spoke.rbvi.ucsf.edu. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

OBJECTIVE: Although social and environmental factors are central to provider-patient interactions, the data that reflect these factors can be incomplete, vague, and subjective. We sought to create a conceptual framework to describe and classify data about presence, the domain of interpersonal connection in medicine. METHODS: Our top-down approach for ontology development based on the concept of "relationality" included the following: 1) a broad survey of the social sciences literature and a systematic literature review of >20 000 articles around interpersonal connection in medicine, 2) relational ethnography of clinical encounters (n = 5 pilot, 27 full), and 3) interviews about relational work with 40 medical and nonmedical professionals. We formalized the model using the Web Ontology Language in the Protégé ontology editor. We iteratively evaluated and refined the Presence Ontology through manual expert review and automated annotation of literature. RESULTS AND DISCUSSION: The Presence Ontology facilitates the naming and classification of concepts that would otherwise be vague. Our model categorizes contributors to healthcare encounters and factors such as communication, emotions, tools, and environment. Ontology evaluation indicated that cognitive models (both patients' explanatory models and providers' caregiving approaches) influenced encounters and were subsequently incorporated. We show how ethnographic methods based in relationality can aid the representation of experiential concepts (eg, empathy, trust). Our ontology could support investigative methods to improve healthcare processes for both patients and healthcare providers, including annotation of videotaped encounters, development of clinical instruments to measure presence, or implementation of electronic health record-based reminders for providers. CONCLUSION: The Presence Ontology provides a model for using ethnographic approaches to classify interpersonal data.

OBJECTIVE: To assess usability and usefulness of a machine learning-based order recommender system applied to simulated clinical cases. MATERIALS AND METHODS: 43 physicians entered orders for 5 simulated clinical cases using a clinical order entry interface with or without access to a previously developed automated order recommender system. Cases were randomly allocated to the recommender system in a 3:2 ratio. A panel of clinicians scored whether the orders placed were clinically appropriate. Our primary outcome included the difference in clinical appropriateness scores. Secondary outcomes included total number of orders, case time, and survey responses. RESULTS: Clinical appropriateness scores per order were comparable for cases randomized to the order recommender system (mean difference -0.11 order per score, 95% CI: [-0.41, 0.20]). Physicians using the recommender placed more orders (median 16 vs 15 orders, incidence rate ratio 1.09, 95%CI: [1.01-1.17]). Case times were comparable with the recommender system. Order suggestions generated from the recommender system were more likely to match physician needs than standard manual search options. Physicians used recommender suggestions in 98% of available cases. Approximately 95% of participants agreed the system would be useful for their workflows. DISCUSSION: User testing with a simulated electronic medical record interface can assess the value of machine learning and clinical decision support tools for clinician usability and acceptance before live deployments. CONCLUSIONS: Clinicians can use and accept machine learned clinical order recommendations integrated into an electronic order entry interface in a simulated setting. The clinical appropriateness of orders entered was comparable even when supported by automated recommendations.

Discussions regarding open science have circulated in the scientific community for many years. The articulation of the FAIR principles in 2016, however, led to a groundswell of excitement to make experimental data findable, accessible, interoperable, and reusable. The FAIR acronym is catchy and easy to remember. The 15 FAIR principles, however, are not. Efforts to enhance access to scientific datasets and to promote their reuse require intuitive tools that implement the FAIR principles as a side effect of their use. The CEDAR Workbench is one such tool that simplifies the authoring of standardized, comprehensive metadata to make datasets FAIR. Systems such as the CEDAR Workbench, which renders datasets FAIR in a transparent fashion, can enhance open science as a direct byproduct of their use. Current projects that have adopted the CEDAR Workbench provide an opportunity to assess how well knowledge technologies can facilitate the creation of FAIR data.

The metadata about scientific experiments published in online repositories have been shown to suffer from a high degree of representational heterogeneity—there are often many ways to represent the same type of information, such as a geographical location via its latitude and longitude. To harness the potential that metadata have for discovering scientific data, it is crucial that they be represented in a uniform way that can be queried effectively. One step toward uniformly-represented metadata is to normalize the multiple, distinct field names used in metadata (e.g., lat lon , lat and long ) to describe the same type of value. To that end, we present a new method based on clustering and embeddings (i.e., vector representations of words) to align metadata field names with ontology terms. We apply our method to biomedical metadata by generating embeddings for terms in biomedical ontologies from the BioPortal repository. We carried out a comparative study between our method and the NCBO Annotator, which revealed that our method yields more and substantially better alignments between metadata and ontology terms.

This paper introduces HopRank, an algorithm for modeling human navigation on semantic networks. HopRank leverages the assumption that users know or can see the whole structure of the network. Therefore, besides following links, they also follow nodes at certain distances (i.e., k-hop neighborhoods), and not at random as suggested by PageRank, which assumes only links are known or visible. We observe such preference towards k-hop neighborhoods on BioPortal, one of the leading repositories of biomedical ontologies on the Web. In general, users navigate within the vicinity of a concept. But they also “jump” to distant concepts less frequently. We fit our model on 11 ontologies using the transition matrix of clickstreams, and show that semantic structure can influence teleportation in PageRank. This suggests that users-to some extent-utilize knowledge about the underlying structure of ontologies, and leverage it to reach certain pieces of information. Our results help the development and improvement of user interfaces for ontology exploration.

BACKGROUND: Public biomedical data repositories often provide web-based interfaces to collect experimental metadata. However, these interfaces typically reflect the ad hoc metadata specification practices of the associated repositories, leading to a lack of standardization in the collected metadata. This lack of standardization limits the ability of the source datasets to be broadly discovered, reused, and integrated with other datasets. To increase reuse, discoverability, and reproducibility of the described experiments, datasets should be appropriately annotated by using agreed-upon terms, ideally from ontologies or other controlled term sources. RESULTS: This work presents "CEDAR OnDemand", a browser extension powered by the NCBO (National Center for Biomedical Ontology) BioPortal that enables users to seamlessly enter ontology-based metadata through existing web forms native to individual repositories. CEDAR OnDemand analyzes the web page contents to identify the text input fields and associate them with relevant ontologies which are recommended automatically based upon input fields' labels (using the NCBO ontology recommender) and a pre-defined list of ontologies. These field-specific ontologies are used for controlling metadata entry. CEDAR OnDemand works for any web form designed in the HTML format. We demonstrate how CEDAR OnDemand works through the NCBI (National Center for Biotechnology Information) BioSample web-based metadata entry. CONCLUSION: CEDAR OnDemand helps lower the barrier of incorporating ontologies into standardized metadata entry for public data repositories. CEDAR OnDemand is available freely on the Google Chrome store https://chrome.google.com/webstore/search/CEDAROnDemand.

Integrated approaches for pharmacology are required for the mechanism-based predictions of adverse drug reactions that manifest due to concomitant intake of multiple drugs. These approaches require the integration and analysis of biomedical data and knowledge from multiple, heterogeneous sources with varying schemas, entity notations, and formats. To tackle these integrative challenges, the Semantic Web community has published and linked several datasets in the Life Sciences Linked Open Data (LSLOD) cloud using established W3C standards. We present the PhLeGrA platform for Linked Graph Analytics in Pharmacology in this paper. Through query federation, we integrate four sources from the LSLOD cloud and extract a drug-reaction network, composed of distinct entities. We represent this graph as a hidden conditional random field (HCRF), a discriminative latent variable model that is used for structured output predictions. We calculate the underlying probability distributions in the drug-reaction HCRF using the datasets from the U.S. Food and Drug Administration's Adverse Event Reporting System. We predict the occurrence of 146 adverse reactions due to multiple drug intake with an AUROC statistic greater than 0.75. The PhLeGrA platform can be extended to incorporate other sources published using Semantic Web technologies, as well as to discover other types of pharmacological associations.

Ontologies in the biomedical domain are numerous, highly specialized and very expensive to develop. Thus, a crucial prerequisite for ontology adoption and reuse is effective support for exploring and finding existing ontologies. Towards that goal, the National Center for Biomedical Ontology (NCBO) has developed BioPortal---an online repository containing more than 500 biomedical ontologies. In 2016, BioPortal represents one of the largest portals for exploration of semantic biomedical vocabularies and terminologies, which is used by many researchers and practitioners. While usage of this portal is high, we know very little about how exactly users search and explore ontologies and what kind of usage patterns or user groups exist in the first place. Deeper insights into user behavior on such portals can provide valuable information to devise strategies for a better support of users in exploring and finding existing ontologies, and thereby enable better ontology reuse. To that end, we study and group users according to their browsing behavior on BioPortal and use data mining techniques to characterize and compare exploration strategies across ontologies. In particular, we were able to identify seven distinct browsing types, all relying on different functionality provided by BioPortal. For example, Search Explorers extensively use the search functionality while Ontology Tree Explorers mainly rely on the class hierarchy for exploring ontologies. Further, we show that specific characteristics of ontologies influence the way users explore and interact with the website. Our results may guide the development of more user-oriented systems for ontology exploration on the Web.

Terminologies are typically large and complex knowledge systems. It is difficult to obtain an orientation into their structure and content. In previous research we designed compact summary networks called partial-area taxonomies to provide a structural summary of a terminology. The sizes of a terminology and of its partial-area taxonomy are defined as their numbers of nodes. While a partial-area taxonomy is typically smaller than the original terminology, it is often not compact enough to provide a clear “big picture,” due to too many nodes that summarize only a small number of terminology concepts. The display of such a partial-area taxonomy is still overwhelming. In this paper, we introduce a more compact summary of a terminology, called an aggregate taxonomy, obtained by aggregating small partial-area taxonomy nodes into larger nodes. We present a parametrized technique to study the design of such an aggregate taxonomy and apply it to the Specimen hierarchy of SNOMED CT. A software tool for creating and displaying aggregate taxonomies is described. We illustrate how aggregate taxonomies derived across multiple SNOMED CT releases can be used to summarize the evolution of the Specimen hierarchy's content over eight years of SNOMED CT releases.

An ontology is a machine processable artifact that captures knowledge about some domain of interest. Ontologies are used in various domains including healthcare, science, and commerce. In this paper we examine the ontology bootstrapping problem. Specifically, we look at an approach that uses both competency questions and knowledge source reuse via recommendations to address the "cold start problem" that is, the task of creating an ontology from scratch. We describe this approach, an implementation of it, and we present an evaluation in the form of a controlled user study. We find that the approach leads users into creating significantly more detailed initial ontologies that have a greater domain coverage than ontologies produced without this support. Furthermore, in spite of a more involved workflow, the usability and user satisfaction of the bootstrapping approach is as good as a state-of-the-art ontology editor with no additional support.

OBJECTIVE: The capability to share data, and harness its potential to generate knowledge rapidly and inform decisions, can have transformative effects that improve health. The infrastructure to achieve this goal at scale--marrying technology, process, and policy--is commonly referred to as the Learning Health System (LHS). Achieving an LHS raises numerous scientific challenges. MATERIALS AND METHODS: The National Science Foundation convened an invitational workshop to identify the fundamental scientific and engineering research challenges to achieving a national-scale LHS. The workshop was planned by a 12-member committee and ultimately engaged 45 prominent researchers spanning multiple disciplines over 2 days in Washington, DC on 11-12 April 2013. RESULTS: The workshop participants collectively identified 106 research questions organized around four system-level requirements that a high-functioning LHS must satisfy. The workshop participants also identified a new cross-disciplinary integrative science of cyber-social ecosystems that will be required to address these challenges. CONCLUSIONS: The intellectual merit and potential broad impacts of the innovations that will be driven by investments in an LHS are of great potential significance. The specific research questions that emerged from the workshop, alongside the potential for diverse communities to assemble to address them through a 'new science of learning systems', create an important agenda for informatics and related disciplines.

OBJECTIVES: The verification of biomedical ontologies is an arduous process that typically involves peer review by subject-matter experts. This work evaluated the ability of crowdsourcing methods to detect errors in SNOMED CT (Systematized Nomenclature of Medicine Clinical Terms) and to address the challenges of scalable ontology verification. METHODS: We developed a methodology to crowdsource ontology verification that uses micro-tasking combined with a Bayesian classifier. We then conducted a prospective study in which both the crowd and domain experts verified a subset of SNOMED CT comprising 200 taxonomic relationships. RESULTS: The crowd identified errors as well as any single expert at about one-quarter of the cost. The inter-rater agreement (κ) between the crowd and the experts was 0.58; the inter-rater agreement between experts themselves was 0.59, suggesting that the crowd is nearly indistinguishable from any one expert. Furthermore, the crowd identified 39 previously undiscovered, critical errors in SNOMED CT (eg, 'septic shock is a soft-tissue infection'). DISCUSSION: The results show that the crowd can indeed identify errors in SNOMED CT that experts also find, and the results suggest that our method will likely perform well on similar ontologies. The crowd may be particularly useful in situations where an expert is unavailable, budget is limited, or an ontology is too large for manual error checking. Finally, our results suggest that the online anonymous crowd could successfully complete other domain-specific tasks. CONCLUSIONS: We have demonstrated that the crowd can address the challenges of scalable ontology verification, completing not only intuitive, common-sense tasks, but also expert-level, knowledge-intensive tasks.

The Center for Expanded Data Annotation and Retrieval is studying the creation of comprehensive and expressive metadata for biomedical datasets to facilitate data discovery, data interpretation, and data reuse. We take advantage of emerging community-based standard templates for describing different kinds of biomedical datasets, and we investigate the use of computational techniques to help investigators to assemble templates and to fill in their values. We are creating a repository of metadata from which we plan to identify metadata patterns that will drive predictive data entry when filling in metadata templates. The metadata repository not only will capture annotations specified when experimental datasets are initially created, but also will incorporate links to the published literature, including secondary analyses and possible refinements or retractions of experimental interpretations. By working initially with the Human Immunology Project Consortium and the developers of the ImmPort data repository, we are developing and evaluating an end-to-end solution to the problems of metadata authoring and management that will generalize to other data-management environments.

UNLABELLED: WebProtégé is an open-source Web application for editing OWL 2 ontologies. It contains several features to aid collaboration, including support for the discussion of issues, change notification and revision-based change tracking. WebProtégé also features a simple user interface, which is geared towards editing the kinds of class descriptions and annotations that are prevalent throughout biomedical ontologies. Moreover, it is possible to configure the user interface using views that are optimized for editing Open Biomedical Ontology (OBO) class descriptions and metadata. Some of these views are shown in the Supplementary Material and can be seen in WebProtégé itself by configuring the project as an OBO project. AVAILABILITY AND IMPLEMENTATION: WebProtégé is freely available for use on the Web at http://webprotege.stanford.edu. It is implemented in Java and JavaScript using the OWL API and the Google Web Toolkit. All major browsers are supported. For users who do not wish to host their ontologies on the Stanford servers, WebProtégé is available as a Web app that can be run locally using a Servlet container such as Tomcat. Binaries, source code and documentation are available under an open-source license at http://protegewiki.stanford.edu/wiki/WebProtege.

With the emergence of tools for collaborative ontology engineering, more and more data about the creation process behind collaborative construction of ontologies is becoming available. Today, collaborative ontology engineering tools such as Collaborative Protégé offer rich and structured logs of changes, thereby opening up new challenges and opportunities to study and analyze the creation of collaboratively constructed ontologies. While there exists a plethora of visualization tools for ontologies, they have primarily been built to visualize aspects of the final product (the ontology) and not the collaborative processes behind construction (e.g. the changes made by contributors over time). To the best of the authors’ knowledge, there exists no ontology visualization tool today that focuses primarily on visualizing the history behind collaboratively constructed ontologies. Since the ontology engineering processes can influence the quality of the final ontology, they believe that visualizing process data represents an important stepping-stone towards better understanding of managing the collaborative construction of ontologies in the future. In this application paper, the authors present a tool – PragmatiX – which taps into structured change logs provided by tools such as Collaborative Protégé to visualize various pragmatic aspects of collaborative ontology engineering. The tool is aimed at managers and leaders of collaborative ontology engineering projects to help them in monitoring progress, in exploring issues and problems, and in tracking quality-related issues such as overrides and coordination among contributors. The paper makes the following contributions: (i) They present PragmatiX, a tool for visualizing the creation process behind collaboratively constructed ontologies (ii) the authors illustrate the functionality and generality of the tool by applying it to structured logs of changes of two large collaborative ontology-engineering projects and (iii) they conduct a heuristic evaluation of the tool with domain experts to uncover early design challenges and opportunities for improvement. Finally, the authors hope that this work sparks a new line of research on visualization tools for collaborative ontology engineering projects.

The AMIA biomedical informatics (BMI) core competencies have been designed to support and guide graduate education in BMI, the core scientific discipline underlying the breadth of the field's research, practice, and education. The core definition of BMI adopted by AMIA specifies that BMI is 'the interdisciplinary field that studies and pursues the effective uses of biomedical data, information, and knowledge for scientific inquiry, problem solving and decision making, motivated by efforts to improve human health.' Application areas range from bioinformatics to clinical and public health informatics and span the spectrum from the molecular to population levels of health and biomedicine. The shared core informatics competencies of BMI draw on the practical experience of many specific informatics sub-disciplines. The AMIA BMI analysis highlights the central shared set of competencies that should guide curriculum design and that graduate students should be expected to master.

The National Center for Biomedical Ontology is now in its seventh year. The goals of this National Center for Biomedical Computing are to: create and maintain a repository of biomedical ontologies and terminologies; build tools and web services to enable the use of ontologies and terminologies in clinical and translational research; educate their trainees and the scientific community broadly about biomedical ontology and ontology-based technology and best practices; and collaborate with a variety of groups who develop and use ontologies and terminologies in biomedicine. The centerpiece of the National Center for Biomedical Ontology is a web-based resource known as BioPortal. BioPortal makes available for research in computationally useful forms more than 270 of the world's biomedical ontologies and terminologies, and supports a wide range of web services that enable investigators to use the ontologies to annotate and retrieve data, to generate value sets and special-purpose lexicons, and to perform advanced analytics on a wide range of biomedical data.

OBJECTIVE: To characterise empirical instances of Unified Medical Language System (UMLS) Metathesaurus term strings in a large clinical corpus, and to illustrate what types of term characteristics are generalisable across data sources. DESIGN: Based on the occurrences of UMLS terms in a 51 million document corpus of Mayo Clinic clinical notes, this study computes statistics about the terms' string attributes, source terminologies, semantic types and syntactic categories. Term occurrences in 2010 i2b2/VA text were also mapped; eight example filters were designed from the Mayo-based statistics and applied to i2b2/VA data. RESULTS: For the corpus analysis, negligible numbers of mapped terms in the Mayo corpus had over six words or 55 characters. Of source terminologies in the UMLS, the Consumer Health Vocabulary and Systematized Nomenclature of Medicine-Clinical Terms (SNOMED-CT) had the best coverage in Mayo clinical notes at 106426 and 94788 unique terms, respectively. Of 15 semantic groups in the UMLS, seven groups accounted for 92.08% of term occurrences in Mayo data. Syntactically, over 90% of matched terms were in noun phrases. For the cross-institutional analysis, using five example filters on i2b2/VA data reduces the actual lexicon to 19.13% of the size of the UMLS and only sees a 2% reduction in matched terms. CONCLUSION: The corpus statistics presented here are instructive for building lexicons from the UMLS. Features intrinsic to Metathesaurus terms (well formedness, length and language) generalise easily across clinical institutions, but term frequencies should be adapted with caution. The semantic groups of mapped terms may differ slightly from institution to institution, but they differ greatly when moving to the biomedical literature domain.

Advanced statistical methods used to analyze high-throughput data such as gene-expression assays result in long lists of "significant genes." One way to gain insight into the significance of altered expression levels is to determine whether Gene Ontology (GO) terms associated with a particular biological process, molecular function, or cellular component are over- or under-represented in the set of genes deemed significant. This process, referred to as enrichment analysis, profiles a gene-set, and is widely used to makes sense of the results of high-throughput experiments. The canonical example of enrichment analysis is when the output dataset is a list of genes differentially expressed in some condition. To determine the biological relevance of a lengthy gene list, the usual solution is to perform enrichment analysis with the GO. We can aggregate the annotating GO concepts for each gene in this list, and arrive at a profile of the biological processes or mechanisms affected by the condition under study. While GO has been the principal target for enrichment analysis, the methods of enrichment analysis are generalizable. We can conduct the same sort of profiling along other ontologies of interest. Just as scientists can ask "Which biological process is over-represented in my set of interesting genes or proteins?" we can also ask "Which disease (or class of diseases) is over-represented in my set of interesting genes or proteins?". For example, by annotating known protein mutations with disease terms from the ontologies in BioPortal, Mort et al. recently identified a class of diseases--blood coagulation disorders--that were associated with a 14-fold depletion in substitutions at O-linked glycosylation sites. With the availability of tools for automatic annotation of datasets with terms from disease ontologies, there is no reason to restrict enrichment analyses to the GO. In this chapter, we will discuss methods to perform enrichment analysis using any ontology available in the biomedical domain. We will review the general methodology of enrichment analysis, the associated challenges, and discuss the novel translational analyses enabled by the existence of public, national computational infrastructure and by the use of disease ontologies in such analyses.

Scientists and researchers often use ontologies to describe their data, to share and integrate this data from heterogeneous sources. Ontologies are formal computer models that describe the main concepts and their relationships in a particular domain. Ontologies are usually authored by a community of users with different roles and levels of expertise. To support collaboration among distributed teams and to provision for distinct authoring requirements of each of the user roles and of individual users, we designed a configurable Web-based ontology editor, WebProtege. WebProtege extends Protege, a widely popular ontology editor with more than 150,000 registered users. The user interface layout and configuration for WebProtege is model-based and declarative: we represent it in a knowledge base, with an ontology defining its structure, and linking the interface configuration to the users, their roles, and access policies. We will discuss how the knowledge base driven configuration of the user interface supports the reuse and modularization of layout configurations. Such configuration is also highly flexible and extensible, and is easier to manage than many traditional approaches.

The problem of ontology modularization is an active area of research in the Semantic Web community. With the emergence and wider use of very large ontologies, in particular in fields such as biomedicine, more and more application developers need to extract meaningful modules of these ontologies to use in their applications. Researchers have also noted that many ontology-maintenance tasks would be simplified if we could extract modules from ontologies. These tasks include ontology matching: If we can separate ontologies into modules based on the topics that these modules cover, we can simplify and improve ontology matching. In this paper, we study a complementary problem: Can we use existing mappings between ontologies to facilitate modularization? We present a novel approach to modularization based on mappings between ontologies. We validate and analyze our approach by applying our methods to identify modules for National Cancer Institutes Thesaurus (NCI Thesaurus) and Systematized Nomenclature of Medicine--Clinical Terms (SNOMED-CT).

The biomedical research community relies on a diverse set of resources, both within their own institutions and at other research centers. In addition, an increasing number of shared electronic resources have been developed. Without effective means to locate and query these resources, it is challenging, if not impossible, for investigators to be aware of the myriad resources available, or to effectively perform resource discovery when the need arises. In this paper, we describe the development and use of the Biomedical Resource Ontology (BRO) to enable semantic annotation and discovery of biomedical resources. We also describe the Resource Discovery System (RDS) which is a federated, inter-institutional pilot project that uses the BRO to facilitate resource discovery on the Internet. Through the RDS framework and its associated Biositemaps infrastructure, the BRO facilitates semantic search and discovery of biomedical resources, breaking down barriers and streamlining scientific research that will improve human health.

Cloud-based systems have proven to be a powerful technology for building data-intensive applications. However, the process of designing and deploying such applications is still primarily a manual one. There is a need for mechanisms and tools to help automate the required development steps. Using the Semantic Web ontology language OWL and the Hadoop platform we have developed a number of models and associated software tools that provide an end-to-end solution for designing and deploying cloud-based systems. This solution supports the construction of detailed models of data dependencies and their validation. It also enables generation and deployment of cloud-based data flows from those models. We illustrate its use for detecting alarm scenarios using data from vast underwater sensor-network.

BACKGROUND: A critical challenge in neuroscience is organizing, managing, and accessing the explosion in neuroscientific knowledge, particularly anatomic knowledge. We believe that explicit knowledge-based approaches to make neuroscientific knowledge computationally accessible will be helpful in tackling this challenge and will enable a variety of applications exploiting this knowledge, such as surgical planning. RESULTS: We developed ontology-based models of neuroanatomy to enable symbolic lookup, logical inference and mathematical modeling of neural systems. We built a prototype model of the motor system that integrates descriptive anatomic and qualitative functional neuroanatomical knowledge. In addition to modeling normal neuroanatomy, our approach provides an explicit representation of abnormal neural connectivity in disease states, such as common movement disorders. The ontology-based representation encodes both structural and functional aspects of neuroanatomy. The ontology-based models can be evaluated computationally, enabling development of automated computer reasoning applications. CONCLUSION: Neuroanatomical knowledge can be represented in machine-accessible format using ontologies. Computational neuroanatomical approaches such as described in this work could become a key tool in translational informatics, leading to decision support applications that inform and guide surgical planning and personalized care for neurological disease in the future.

The National Center for Biomedical Ontology (NCBO) is developing a system for automated, ontology-based access to online biomedical resources (Shah NH, et al.: Ontology-driven indexing of public datasets for translational bioinformatics. BMC Bioinformatics 2009, 10(Suppl 2):S1). The system's indexing workflow processes the text metadata of diverse resources such as datasets from GEO and ArrayExpress to annotate and index them with concepts from appropriate ontologies. This indexing requires the use of a concept-recognition tool to identify ontology concepts in the resource's textual metadata. In this paper, we present a comparison of two concept recognizers - NLM's MetaMap and the University of Michigan's Mgrep. We utilize a number of data sources and dictionaries to evaluate the concept recognizers in terms of precision, recall, speed of execution, scalability and customizability. Our evaluations demonstrate that Mgrep has a clear edge over MetaMap for large-scale service oriented applications. Based on our analysis we also suggest areas of potential improvements for Mgrep. We have subsequently used Mgrep to build the Open Biomedical Annotator service. The Annotator service has access to a large dictionary of biomedical terms derived from the United Medical Language System (UMLS) and NCBO ontologies. The Annotator also leverages the hierarchical structure of the ontologies and their mappings to expand annotations. The Annotator service is available to the community as a REST Web service for creating ontology-based annotations of their data.

The volume of publicly available genomic scale data is increasing. Genomic datasets in public repositories are annotated with free-text fields describing the pathological state of the studied sample. These annotations are not mapped to concepts in any ontology, making it difficult to integrate these datasets across repositories. We have previously developed methods to map text-annotations of tissue microarrays to concepts in the NCI thesaurus and SNOMED-CT. In this work we generalize our methods to map text annotations of gene expression datasets to concepts in the UMLS. We demonstrate the utility of our methods by processing annotations of datasets in the Gene Expression Omnibus. We demonstrate that we enable ontology-based querying and integration of tissue and gene expression microarray data. We enable identification of datasets on specific diseases across both repositories. Our approach provides the basis for ontology-driven data integration for translational research on gene and protein expression data. Based on this work we have built a prototype system for ontology based annotation and indexing of biomedical data. The system processes the text metadata of diverse resource elements such as gene expression data sets, descriptions of radiology images, clinical-trial reports, and PubMed article abstracts to annotate and index them with concepts from appropriate ontologies. The key functionality of this system is to enable users to locate biomedical data resources related to particular ontology concepts.

OBJECTIVE: Statistical aberrancy-detection algorithms play a central role in automated public health systems, analyzing large volumes of clinical and administrative data in real-time with the goal of detecting disease outbreaks rapidly and accurately. Not all algorithms perform equally well in terms of sensitivity, specificity, and timeliness in detecting disease outbreaks and the evidence describing the relative performance of different methods is fragmented and mainly qualitative. DESIGN: We developed and evaluated a unified model of aberrancy-detection algorithms and a software infrastructure that uses this model to conduct studies to evaluate detection performance. We used a task-analytic methodology to identify the common features and meaningful distinctions among different algorithms and to provide an extensible framework for gathering evidence about the relative performance of these algorithms using a number of evaluation metrics. We implemented our model as part of a modular software infrastructure (Biological Space-Time Outbreak Reasoning Module, or BioSTORM) that allows configuration, deployment, and evaluation of aberrancy-detection algorithms in a systematic manner. MEASUREMENT: We assessed the ability of our model to encode the commonly used EARS algorithms and the ability of the BioSTORM software to reproduce an existing evaluation study of these algorithms. RESULTS: Using our unified model of aberrancy-detection algorithms, we successfully encoded the EARS algorithms, deployed these algorithms using BioSTORM, and were able to reproduce and extend previously published evaluation results. CONCLUSION: The validated model of aberrancy-detection algorithms and its software implementation will enable principled comparison of algorithms, synthesis of results from evaluation studies, and identification of surveillance algorithms for use in specific public health settings.

Functional brain networks detected in task-free ("resting-state") functional magnetic resonance imaging (fMRI) have a small-world architecture that reflects a robust functional organization of the brain. Here, we examined whether this functional organization is disrupted in Alzheimer's disease (AD). Task-free fMRI data from 21 AD subjects and 18 age-matched controls were obtained. Wavelet analysis was applied to the fMRI data to compute frequency-dependent correlation matrices. Correlation matrices were thresholded to create 90-node undirected-graphs of functional brain networks. Small-world metrics (characteristic path length and clustering coefficient) were computed using graph analytical methods. In the low frequency interval 0.01 to 0.05 Hz, functional brain networks in controls showed small-world organization of brain activity, characterized by a high clustering coefficient and a low characteristic path length. In contrast, functional brain networks in AD showed loss of small-world properties, characterized by a significantly lower clustering coefficient (p<0.01), indicative of disrupted local connectivity. Clustering coefficients for the left and right hippocampus were significantly lower (p<0.01) in the AD group compared to the control group. Furthermore, the clustering coefficient distinguished AD participants from the controls with a sensitivity of 72% and specificity of 78%. Our study provides new evidence that there is disrupted organization of functional brain networks in AD. Small-world metrics can characterize the functional organization of the brain in AD, and our findings further suggest that these network measures may be useful as an imaging-based biomarker to distinguish AD from healthy aging.

As more ontologies become publicly available, finding the "right" ontologies becomes much harder. In this paper, we address the problem of ontology search: finding a collection of ontologies from an ontology repository that are relevant to the user's query. In particular, we look at the case when users search for ontologies relevant to a particular topic (e.g., an ontology about anatomy). Ontologies that are most relevant to such query often do not have the query term in the names of their concepts (e.g., the Foundational Model of Anatomy ontology does not have the term "anatomy" in any of its concepts' names). Thus, we present a new ontology-search technique that helps users in these types of searches. When looking for ontologies on a particular topic (e.g., anatomy), we retrieve from the Web a collection of terms that represent the given domain (e.g., terms such as body, brain, skin, etc. for anatomy). We then use these terms to expand the user query. We evaluate our algorithm on queries for topics in the biomedical domain against a repository of biomedical ontologies. We use the results obtained from experts in the biomedical-ontology domain as the gold standard. Our experiments demonstrate that using our method for query expansion improves retrieval results by a 113%, compared to the tools that search only for the user query terms and consider only class and property names (like Swoogle). We show 43% improvement for the case where not only class and property names but also property values are taken into account.

UNLABELLED: The Open Biomedical Ontologies (OBO) format from the GO consortium is a very successful format for biomedical ontologies, including the Gene Ontology. But it lacks formal computational definitions for its constructs and tools, like DL reasoners, to facilitate ontology development/maintenance. We describe the OBO Converter, a Java tool to convert files from OBO format to Web Ontology Language (OWL) (and vice versa) that can also be used as a Protégé Tab plug-in. It uses the OBO to OWL mapping provided by the National Center for Biomedical Ontologies (NCBO) (a joint effort of OBO developers and OWL experts) and offers options to ease the task of saving/reading files in both formats. AVAILABILITY: bioontology.org/tools/oboinowl/obo_converter.html. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

BACKGROUND: The Stanford Tissue Microarray Database (TMAD) is a repository of data serving a consortium of pathologists and biomedical researchers. The tissue samples in TMAD are annotated with multiple free-text fields, specifying the pathological diagnoses for each sample. These text annotations are not structured according to any ontology, making future integration of this resource with other biological and clinical data difficult. RESULTS: We developed methods to map these annotations to the NCI thesaurus. Using the NCI-T we can effectively represent annotations for about 86% of the samples. We demonstrate how this mapping enables ontology driven integration and querying of tissue microarray data. We have deployed the mapping and ontology driven querying tools at the TMAD site for general use. CONCLUSION: We have demonstrated that we can effectively map the diagnosis-related terms describing a sample in TMAD to the NCI-T. The NCI thesaurus terms have a wide coverage and provide terms for about 86% of the samples. In our opinion the NCI thesaurus can facilitate integration of this resource with other biological data.

The SAGE (Standards-Based Active Guideline Environment) project was formed to create a methodology and infrastructure required to demonstrate integration of decision-support technology for guideline-based care in commercial clinical information systems. This paper describes the development and innovative features of the SAGE Guideline Model and reports our experience encoding four guidelines. Innovations include methods for integrating guideline-based decision support with clinical workflow and employment of enterprise order sets. Using SAGE, a clinician informatician can encode computable guideline content as recommendation sets using only standard terminologies and standards-based patient information models. The SAGE Model supports encoding large portions of guideline knowledge as re-usable declarative evidence statements and supports querying external knowledge sources.

Information technology can support the implementation of clinical research findings in practice settings. Technology can address the quality gap in health care by providing automated decision support to clinicians that integrates guideline knowledge with electronic patient data to present real-time, patient-specific recommendations. However, technical success in implementing decision support systems may not translate directly into system use by clinicians. Successful technology integration into clinical work settings requires explicit attention to the organizational context. We describe the application of a "sociotechnical" approach to integration of ATHENA DSS, a decision support system for the treatment of hypertension, into geographically dispersed primary care clinics. We applied an iterative technical design in response to organizational input and obtained ongoing endorsements of the project by the organization's administrative and clinical leadership. Conscious attention to organizational context at the time of development, deployment, and maintenance of the system was associated with extensive clinician use of the system.

No abstract available.

Quality assessment of clinician actions and patient outcomes is a central problem in guideline- or standards-based medical care. In this paper we describe an approach for evaluating and consistently scoring clinician adherence to medical guidelines using the intentions of guideline authors. We present the Quality Indicator Language (QUIL) that may be used to formally specify quality constraints on physician behavior and patient outcomes derived from medical guidelines. We present a modeling and scoring methodology for consistently evaluating multi-step and multi-choice guideline plans based on guideline intentions and their revisions.

The Institute of Medicine recently issued a landmark report on medical error. 1 In the penumbra of this report, every aspect of health care is subject to new scrutiny regarding patient safety. Informatics technology can support patient safety by correcting problems inherent in older technology; however, new information technology can also contribute to new sources of error. We report here a categorization of possible errors that may arise in deploying a system designed to give guideline-based advice on prescribing drugs, an approach to anticipating these errors in an automated guideline system, and design features to minimize errors and thereby maximize patient safety. Our guideline implementation system, based on the EON architecture, provides a framework for a knowledge base that is sufficiently comprehensive to incorporate safety information, and that is easily reviewed and updated by clinician-experts.

The vision of integrating information-from a variety of sources, into the way people work, to improve decisions and process-is one of the cornerstones of biomedical informatics. Thoughts on how this vision might be realized have evolved as improvements in information and communication technologies, together with discoveries in biomedical informatics, and have changed the art of the possible. This review identified three distinct generations of "integration" projects. First-generation projects create a database and use it for multiple purposes. Second-generation projects integrate by bringing information from various sources together through enterprise information architecture. Third-generation projects inter-relate disparate but accessible information sources to provide the appearance of integration. The review suggests that the ideas developed in the earlier generations have not been supplanted by ideas from subsequent generations. Instead, the ideas represent a continuum of progress along the three dimensions of workflow, structure, and extraction.

This is the edited transcript of a discussion, among the authors and audience, that followed the presentation that led to the paper “Integration and Beyond: Linking Information from Disparate Sources and into Workflow,” which appears on p. 135. Mark Musen: Bill, when you presented the three generations of integration, the implication was that the third generation is at hand. It was all present tense. I think all of us agree that architectures that allow us to encapsulate knowledge and data in ways that permit reuse are quite exciting. But are we really in the present tense? Have we really achieved these kinds of architectures and, in particular, when you go to the vendor demonstrations, what do you see of this? Bill Stead: That is a very interesting question, because I think the third generation is more in hand than the second. I think “generation” may be the wrong word, because it suggests that the third supplants the second. Instead, techniques from each of the generations coexist in equilibrium. For example, the UMLS provides us with mapping between codes for the kind of things you order for a patient (diagnosis, tests, medications) and the literature. At Vanderbilt we use this mapping to let you ask, “What are the references relevant to the things that have been ordered?” So, to that degree, third generation exists. It is the second generation that is really hard, because it requires regularization. There's a difference between the Vanderbilt vegetable and the Columbia MED, in that the Columbia MED relates the source vocabularies of the various feed systems (third generation), whereas the Vanderbilt vegetable tries to build an enterprise-wide source vocabulary that is then reflected back into the source systems, prealigning their vocabularies. Back to your question, 3M is an example of a vendor that has pursued an architectural strategy. Member of the audience: I think one of the toughest things we all have to deal with is updating our dictionaries. In the simplest cases, the name of an organism is changed and we just have to do the maintenance. It is tougher, when, as with Citrobacter, they do genetic studies and say, “Oh, it's really six different organisms, not one.” We have the human genome project coming very quickly. Even that is just the tip of the iceberg. We're not only going to see all the genes; we're then going to see clinical tests based on gene expression. Essentially, you'll be able to look at something on the order of 180,000 gene products and whether they're up or down regulated. How are we going to integrate such an incredible amount of data at a time when we're going to also be changing how we think about these processes? Classification and simple mapping are not going to work, because the lumpers and splitters are going to be arguing furiously on a daily basis. Randy Miller: The problems you mentioned are clearly on the horizon and very important. But at a simple level, people are people and all of what you're talking about doesn't change how people will present to their primary care providers. At least that part of what exists will not get torn apart. I think what you're talking about is very rich, very vast information overlays on top of what we already have. We don't have to throw out what we have, we need to be ready to extend the linkages. How that will be done is an unanswered question that will result in multiple research grants. Bill Hersh: I think you allude to one of the key points, which is structuring the metadata with the right levels of granularity. Clearly, when we find an organism that can't fit in the existing framework, then that's problem. But if we find that an organism just represents a subcategory of others, and if there's a good hierarchic structure, it can be fit in. The same goes, for example, for diabetes. People classify diabetes with this complication and that complication, but often we just want to know whether the patient has diabetes. Again, a good hierarchic metadata structure can overcome some of those problems. I think we also need to recognize some of the practical limitations that face us. There are limits to the accuracy of the information that's in medical records; there are limits to the consistency in which people apply vocabulary terms. Computers can be completely precise in terms of mapping from this to that, but people will continue to have different conceptions of what a “grade II systolic murmur” is. Bill Stead: I agree with both answers, but I want to continue to clarify what we are talking about. We get in trouble because people use words to reduce concepts to something that we can manage in our heads. So we lump, and person A lumps differently from person B. So we are each a “legacy system,” and our information resources have grown from this starting point. I think we need to work at two ends of the spectrum. Whenever possible, capture data according to granular definitions. If we have an organism and we discover that it splits into six organisms, that's actually a very easy problem to solve, as you said. What you've got to do is say, “A is now B, C, and D and it mapped here.” That is straightforward. That's the end of the spectrum where we can stay granular. For example, never store a doctor and the doctor's service as one piece of information. At the other end of the spectrum, where the granular definitions are not obvious, do not try to classify the data. Instead, tag a “clump” of information with metadata. This tagging, together with increasingly sophisticated extraction techniques, will be used to approximate meaning. Over time, we will get to a complete set of coded data by working from the two ends. Mark Musen: I'm not sure that everything will ever be completely coded. Given the fact that the world is continuously changing, I don't think we can assume that Aristotle was correct that eventually there will be a classification that we will all accept. For example, I do not know whether gastric ulcer is an infectious disease or a gastrointestinal disease, and maybe it is both. As we continue to learn more about medicine and as our organizations change out from under us, I think we're going to be in the situation where the way we categorize the world is going to change. This is very hard stuff. Instead of working on the ultimate classification that will have all of the problems of the International Classification of Diseases, we need to build structures that not only allow us to enumerate the kinds of data that our programs operate on, but attempt as best as we can to enumerate the assumptions that we're making about our data and about the world. Then, as things change, we can, as human beings, try to update our ontologies. I think we have to be able to deal with changing worlds and with the fact that people and computers each need different views on the data, and that means different assumptions as well. Bill Hersh: To reiterate Mark's point, some people have heard this quote, that “perfect is the enemy of good.” We, especially us academic types, strive for perfection, but in reality the world is not perfect, and I don't know that everything will be perfectly coded. But we can reach compromises, such that we can code bits of information that enable us to do useful things. Bill Stead: I think human beings are each different, but we have an underlying genetic code that we are in the process of discovering. Next, we are going to have to work out the problem of going from genotype to phenotype. When I say that I think in the end things will be coded, I think we're going to discover something that is to information what DNA is to people. It will be a very granular base set of building blocks, which will be rolled up into concepts much as genes produce proteins. So I do not want to go to one ontology or one classification. Still, I like having ontologies, particularly ones that clearly represent the difference between themselves and the others. Member of the audience: I'd like to ask a question about capturing ontologies from multiple people. Imagine for a moment that knowledge freezes long enough for us to try to catch it. Do you have a vision of a tool that will allow multiple knowledge-domain people to act at once? To work out discrepancies in their visions? Mark Musen: Put differently, the question was how do we deal with the fact that there is no overarching ontology? How do we build the tools that will allow us to try to achieve consensus in ontologies? I think the answer to that question is that we do not know. I'm being a little bit facetious, but philosophers have been trying to deal with that problem for 2,000 to 3,000 years. I think you see two different approaches in the computer science community. You see the approach that Doug Lenat has taken. He is trying to create an ontology that he believes will provide all the knowledge that one needs to read the Encyclopaedia Britannica. Such an overarching ontology would need to capture most of human existence. The real problem, though, is how you ever validate the distinctions made in that ontology and have confidence that things have been captured in a way that is consistent and understandable? How do you record all the assumptions that you make while constructing the ontology? When you have concepts like “semi-tangible object” and “semi-intangible object,” it's very hard to know for sure whether what one records about those distinctions really makes sense. At the other end of the spectrum, you see people who really want a thousand flowers to bloom and who are not trying to achieve that kind of perfect alignment among views of the world. For example, the Knowledge Systems Laboratory at Stanford is trying to make constrained ontologies that deal with very narrow domains, so that the kinds of problems that you allude to do not happen, because the number of concepts in the ontology is relatively small. The answer lies somewhere between Doug Lenat's view of the world, that all we have to do is work hard enough and everything will fall into place, and the view that we can't possibly do this, so we have to have just a small number of constrained ontologies. We need to elucidate a set of principles that will provide the basis for tools that will help us try to, if not merge small ontologies, at least create the kinds of alignments that will allow us to bring them together in ways that make them useful. Randy Miller: One of the things that I learned from my mentor, Jack Myers, is that as an informatician, as opposed to a philosopher or a computer scientist, you do not need to represent everything. If you have a problem at hand, you represent it at a level that is tractable and doable. If you do what Doug Lenat's doing, you can spend your entire career representing stuff that is not ever going to be used in a real system, because there is no way to apply it. While that may sound harsh, the reality is that we do not know how to represent time, severity of finding, and severity of illness well at all, but we can still build systems that do diagnosis or a good job of making recommendations for therapy. So you do not have to capture the world in all its infinite detail. The trick is to understand what the critical information is and represent things at that level. Otherwise, you get mired in detail. Mark Musen: Let me underscore your last point. Doug Lenat actually felt pretty confident that his ontology covered all the areas that one would want to deal with, until last year, when HotBot contracted to use CYC as the basis for indexing Web pages. This contract showed, first of all, that ontologies have incredible commercial potential, but it also pointed out to Doug Lenat that there was a whole realm of human experience that was not well represented in the ontology. Specifically, there was a need to categorize different kinds of pornography, which Lenat had not thought about previously. Member of the audience: Health Level Seven's development of a set of reference information models is one of the major efforts for creating a structure for ontologies in the United States. Can you talk about how your organizations are participating in the development of that reference information model (RIM) and how you are using your academic experiences to contribute to that effort among providers, academics, and vendors? Bill Stead: Vanderbilt is an institutional member and a strong advocate of HL7. The central core of our communication subsystem uses HL7, and we build middle ware as needed to bridge between the core and legacy products. We have not put direct energy into the process for defining the reference information model. We use the HL7 model as a starting point, but we extend it as needed. In this way we incorporate it into immediate solutions to real problems, while providing useful information about future directions. Bill Hersh: None of us has been involved directly in that effort. However, our research into the nature of ontologies and the vocabulary projects such as the Cannon Grouping should useful to the effort. Mark Musen: I will just add that I think the vendor community is in the best position to work on ontology content, because they have the most direct connection with the needs of end users. I think that academicians need to follow this work very carefully. We are, we hope, in the best position to be developing the kinds of tools that will help us examine ontologies, relate them to each other, and allow them to evolve as our understanding of the world changes. Randy Miller: I have a slightly contrary view, partly out of ignorance about HL7 RIM. The key question is what problems it is trying to solve. That should drive what the content is. If you can state the problems it is going to be used to solve, then you can say whether it should clinically rich. In that case it will require lots of input from academic clinicians. If it is to solve the problem of interchange of data among vendors, then it needs vendor input. But until you explicitly state what it's going to be used for, just building it for the sake of building it is not useful. I know that the HL7 RIM is not being built that way. I am just saying that I think that's the way to address your question, to seek the specific purpose before giving an answer.

Computer-based systems that support health care require large controlled terminologies to manage names and meanings of data elements. These terminologies are not static, because change in health care is inevitable. To share data and applications in health care, we need standards not only for terminologies and concept representation, but also for representing change. To develop a principled approach to managing change, we analyze the requirements of controlled medical terminologies and consider features that frame knowledge-representation systems have to offer. Based on our analysis, we present a concept model, a set of change operations, and a change-documentation model that may be appropriate for controlled terminologies in health care. We are currently implementing our modeling approach within a computational architecture.

The manner in which a knowledge-acquisition tool displays the contents of a knowledge base affects the way users interact with the system. Previous tools have incorporated semantics that allow knowledge to be edited in terms of either the structural representation of the knowledge or the problem-solving method in which that knowledge is ultimately used. A more effective paradigm may be to use the semantics of the application domain itself to govern access to an expert system's knowledge base. This approach has been explored in a program called OPAL, which allows medical specialists working alone to enter and review cancer treatment plans for use by an expert system called ONCOCIN. Knowledge-acquisition tools based on strong domain models should be useful in application areas whose structure is well understood and for which there is a need for repetitive knowledge entry.

OBJECTIVES: The authors discuss the usability of an automated tool that supports entry, by clinical experts, of the knowledge necessary for forming high-level concepts and patterns from raw time-oriented clinical data. DESIGN: Based on their previous work on the RESUME system for forming high-level concepts from raw time-oriented clinical data, the authors designed a graphical knowledge acquisition (KA) tool that acquires the knowledge required by RESUME. This tool was designed using Protégé, a general framework and set of tools for the construction of knowledge-based systems. The usability of the KA tool was evaluated by three expert physicians and three knowledge engineers in three domains-the monitoring of children's growth, the care of patients with diabetes, and protocol-based care in oncology and in experimental therapy for AIDS. The study evaluated the usability of the KA tool for the entry of previously elicited knowledge. MEASUREMENTS: The authors recorded the time required to understand the methodology and the KA tool and to enter the knowledge; they examined the subjects' qualitative comments; and they compared the output abstractions with benchmark abstractions computed from the same data and a version of the same knowledge entered manually by RESUME experts. RESULTS: Understanding RESUME required 6 to 20 hours (median, 15 to 20 hours); learning to use the KA tool required 2 to 6 hours (median, 3 to 4 hours). Entry times for physicians varied by domain-2 to 20 hours for growth monitoring (median, 3 hours), 6 and 12 hours for diabetes care, and 5 to 60 hours for protocol-based care (median, 10 hours). An increase in speed of up to 25 times (median, 3 times) was demonstrated for all participants when the KA process was repeated. On their first attempt at using the tool to enter the knowledge, the knowledge engineers recorded entry times similar to those of the expert physicians' second attempt at entering the same knowledge. In all cases RESUME, using knowledge entered by means of the KA tool, generated abstractions that were almost identical to those generated using the same knowledge entered manually. CONCLUSION: The authors demonstrate that the KA tool is usable and effective for expert physicians and knowledge engineers to enter clinical temporal-abstraction knowledge and that the resulting knowledge bases are as valid as those produced by manual entry.

By applying recent advances in the standards for distributed computing, we have developed an architecture for a CORBA implementation of a library of platform-independent, sharable problem-solving methods and knowledge bases. The aim of this library is to allow developers to reuse these components across different tasks and domains. Reuse should be cost-effective; therefore, the library will include standard problem-solving methods whose semantics are well understood and are described with a language for stating the requirements and capabilities of a component. In addition, when a developer needs to adapt a component to a new task, the adaptation costs should be minimal. Thus, we advocate the use of separate mediating components that isolate these adaptations from the original component. We demonstrate our approach with an example: an implementation of a problem-solving method, a knowledge-base server, and mediating components that adapt the method to different knowledge bases and tasks.

This paper describes successive reformulations of skeletal-plan refinement as a problem-solving method. We argue that, whereas ideas derived from planning literature helped to determine the overall structure of the planning systems, domain-derived considerations and architectural framework in which the systems were implemented played important roles in these reformulations. We illustrate the argument by describing a new framework that integrates knowledge-based applications with a temporal data-abstraction and data-management system. In this framework, both applications and temporal-data mediators are encapsulated as Common Object Request Broker Architecture (CORBA) objects. The skeletal-plan refinement method itself is formulated as a collection of cooperating CORBA objects. We have found that we needed to reformulate the method ontology, mapping relations and control structure of the skeletal-planning problem-solving method in this framework. Our experience suggests that problem-solving methods are not necessarily fixed structures that can be plugged into arbitrary application environments, and that we need to develop a flexible configuration environment and expressive mapping formalisms to accommodate the requirements of application environments. These requirements include the ways data are made available and the ways software components interact with one another.

The continuing development of the field of medical informatics has raised new questions and placed before us new dilemmas. Spurred by the proliferation of information systems to support the broad missions of our institutions, and the evolution of these systems from luxuries to necessities, organizational issues have assumed increasing prominence. Among a dazzling array of organizational issues now before us is the tension between the long-standing academic role of informatics groups within medical centers and the ever-expanding service role. In the academic role, we seek the knowledge to create improved technology and to train the next generation of informatics researchers. In the service role, we seek to put existing technology, developed internally or purchased from vendors, to best use across the full scope of medical center activities. The dilemma before us is not whether both roles are important—the answer to that is clear—but rather how to organize ourselves within our institutions to address both of them. How much organizational distance should exist between the people who carry out these different roles, and who should direct their efforts? Most academic medical centers are actively searching for answers to these organizational questions, and many AMIA members are engaged in this pursuit. The answers obtained will be of profound consequence for our field. The salience of this issue directed its selection as the focus of the ACMI Debate at the closing session of the 1997 AMIA Fall Symposium. The purpose of the debate was not to generate a universal answer, for no such answer exists, but rather to illuminate the many factors that must be considered as our institutions search for an appropriate organizational model. To frame the debate, we intentionally polarized the issue around a specific proposition: Resolved: Academic medical centers should have a single unit responsible for information systems supporting the clinical and academic missions and also should be charged to carry out high-quality education and research in medical informatics. The polarity is such that the affirmative team would argue in favor of one group under one leader who would carry out all roles. The negative team would argue for a significant level of separation. We adapted the standard high school and college debate format to fit the available time and to use competition as a device to promote deeper understanding of key issues. There were no judges and no declared winners. Each team had two members: Warner Slack and William Stead for the affirmative, Mark Frisse and Mark Musen for the negative. The format included eight-minute constructive statements, two-minute cross-examinations, and three-minute closing (rebuttal) statements in this order: First affirmative constructive statement, by Slack Cross-examination of Slack, by Musen First negative constructive statement, by Frisse Cross-examination of Frisse, by Slack Second affirmative constructive statement, by Stead Cross-examination of Stead, by Frisse Second negative constructive statement, by Musen Cross-examination of Musen, by Stead Closing (rebuttal) statements by Frisse, Slack, Musen, and Stead, in that order In preparing this summary, we sought to convey the substance and spirit of the debate in a manner suited to printed text. This narrative follows the order of the debate as it occurred on October 29, 1997, at the AMIA Fall Symposium in Nashville, Tennessee. The constructive statements included here were edited from the notes the debaters used to prepare their statements. The cross-examinations and closing statements were edited from the debate transcripts and retain much of the colloquial language used in the event. We include bibliographic references only to direct quotations and citations used by the debaters themselves. The format of this and any debate, and most notably the polarization of a multifaceted issue, often requires participants to take extreme positions. The debaters' views, in reality, overlap more than this report would suggest. Some of the debaters believed that they could, if asked, argue with equal effectiveness in support of their opponents' position. Statements of the debaters may be at variance with their own personal beliefs and do not represent official policies of the institutions for they the we the 1997 ACMI Academic medical centers in have two with its the one is the medical school with its academic from on on the is the with its from the or on In the are and in the medical as the of and service in the as the in of is the and must the research is in the the academic will with the from time to but as as or in the and the will be and in who are both medical school with their and with their the and the an of is the the and its are to the and their and the The should be a for and the on the the are to the by the the is often In the had the and all often with an that from to the of has and the is in this is more than in clinical The medical school the clinical within their to a The of clinical is both a and a to is for the most the of the In the were in the academic of the but the also a the and how and clinical are for clinical as an academic The was in an academic the but the in the academic their and in the is a different were suited for than for clinical and were the time to the of the in were The or information and or were in this is the in most the clinical if is is to and their often than the systems are of to the and or The to this that have clinical developed their under the of an academic the and of the on the academic clinical and the clinical to the and in the of that the of on to the and with by and in the of In clinical is a medical and in the medical school it in an academic The of an academic of clinical that be with the to do research with in to and systems that for their or to the of systems of that for their academic on that are now systems will their to be responsible for the of the clinical systems used in the of and to the and of clinical in the medical school and and for and academic clinical is to a In summary, as the of in that with the is to the the and the and education the academic clinical and education are the are to as as to and There are for with the academic to clinical of no for with the academic in but not as an academic are the of an academic the medical school the and most of an academic unit such as or clinical to be the of the as a of to in the to and for their to do research both in the as as the In all of these must be in one academic unit for the to be and medical both of for an academic it that an academic unit do a medical would not an an the was the in have any are of to the that is a of a would such as are as academic within the and to to be is an academic that in the of the may be an of are for that on in in the of we all the questions to that of that on with and on our for informatics who to the of information systems in and research the do not the do we from the and they to in this any more than we our from or our from we that our the the is academic as the of we frame our the of debate that have in medical education the of the for medical education before the We our not on the but on the that have to the of the as a and and the of the academic to the of We our on both the high for academic and the more of academic in We these two should and We also frame these different from the of an and from the of an here in the who is engaged in a medical informatics or We these two and us the as we it should In this the is the for a to the in in the that such knowledge will to in technology and will be to the of informatics in the academic medical center is this is by the of in the we is best not by a single unit but by a between an academic unit and a different group the for and support of information The of is a of We many of the for a single unit to be not on on the that this is the only for the research and service to to and to us now to the the organizational has on the of the academic informatics In the should seek an that a to and The is an between the and the one the of academic and in and one will be both for the to and to the This we the academic or is not in in a of and but the to the or the to focus on has In the academic informatics members an between and is by time to and the issues our field. To such an a for the and of information systems in This is not our we take the that of many of the of medical that are as a of and are we that in academic must have a of and In our these one only one should them. one one should train is a most must from To are only they the of a they are not with their who often do not often requires more than one to be us now the issue from a more at the as it is rather than as it should the the are more a and of and many of the medical informatics groups are their systems and by that will not the of but it is will of organizational The these are in is believed to be the best of their but the they from their they if time on do their a to create a research is a in it not that most institutions have a to on the the academic of the clinical and The of our institutions, in often with their in favor of the have with their the by Stead and a it for the academic to in the and do they do of with to the more of information us the from the of a or who is or a in most of us in the preparing for this debate, often of one of our more who how from the to a in the to and on such as these are by and We it is the who these and all or with the from a We who are in this would take on this that most institutions will create a of in both research and institutions in the who have this and it with for more than a that our best informatics research institutions would be of their if they were all information support the institutions that are support for information systems and whether or not these institutions and their are the most for the of that will generate the that will the next generation of We who in to the have at but have on different of us has an focus on medical informatics and most will have an for to of us have from a research to a that have not to a single organizational but to the and between who do the and who in the In summary, we who in to the argue that a single is more than an often to different groups do not We argue that a single of is but it should at the level of our and for a of knowledge information technology on the of responsible for the of the our are to our are to systems and our are engaged in a debate the of academic we in to is it and how a this We that frame the issue this their will be more in with and not in an on the at the for that have in has in clinical to the has in that the and in the within a medical school is not to was to that research and that medical informatics research will be the for clinical not to argue for an a is research in or should the of that research in clinical be to the as it out the have to the is that the is it is for an to the by an in the if we put the research and we are our research at would that two have not research that clinical the in the of a medical a clinical are with the in of this is often the in own as with the the as would be with it is an of The field of medical informatics is at a is We must that we generate that on a systems that or the are and the we will be unit with for and of the and has the best to this Each of these must be but is a and the research a and a of the of the to an that will in research argue that the of research and will be or that will be more be obtained but such are not a of an The from a to support of the of the or from the of a that the and is a but one that must be is no different from how to our missions as we our with their are an a medical a between informatics and the and an to the that it be in is of the In the must use the for or use it before it is for to use it on their In the of the and the be used before the by would be as a by its This is is not a have to to a of informatics and they that they have but that no one would use whether they used in to the they do not a as in clinical is also a to is from informatics. were was of a of a of with a of between a of a to a information systems one of these such as order and to with systems that have placed in such as often To the of and new of that would not be information technology, such as by and it is to out how a new and of information technology would be to the and This be by the informatics research in an in a medical has the and to use a is the on the and the research is no that the of the their clinical systems in an informatics be it is by the that This is for who must in a time The is no for the in technology an if must be to support an be a it is it the it to This of fit requires between responsible for the research and responsible for of has a of in medical with generation and development and to is available for the and is available for the two an to different roles in the and to and between as their the of are in they have a they the of a to it do it to by the of the in the field that was to the informatics is the and knowledge in that to We who both informatics and We who and The that exist will not be the who are responsible for academic informatics with their who are responsible for the and that is much with an organizational In that the was at the of informatics that of the of their clinical occurred on time and and that only of available clinical was used in the that the for is than these the of these statements that the not how to that be in time to our of the that from the to the their will not such an answer from the and of the at We own for the of the we we on it to create a We that level of of our for the of academic medical informatics In do research for do research as as in an that has a that research to be How do by are of a and the at in a if that in the if a that may take to own time is to than it is to are to a if research to support development of that and if do not from the to how to on that have from the for has for to that they are of an unit that also has for one of our informatics to as a unit to an understanding of how the be the was that a not have a that was as a unit was in a to out that the was the role of a unit and that they have such a their were as a and the The to the of the medical informatics research they do not have to that that the of academic medical centers are that academic is on all the of the model. we are on for such as and clinical such as an that informatics to be that the to and to and is by a to the one not a model. is a of team a under is no at do not of as a We are here to academic medical informatics. is not to the of clinical in academic medical are significant and we are of the issues that Stead out in We are to clinical We are we are not here to debate how best to address the of information in academic medical we are here to debate how best to address the of medical informatics as an academic Mark Frisse and that medical informatics has both an academic and a service two are in their In the that the affirmative team a single has for and research in medical informatics and for the of the medical is increasing clinical the academic and the service of medical informatics must more is our that academic research and service have their own and that the and for in one not by any that one in the is to take a and is all in this debate, Warner Slack to the of an academic unit in a school of Warner that to include clinical and Stead as an of an academic the affirmative team these they to the of how academic in medical informatics best their for research and The in this debate has on and on clinical systems in these are for medical they are from the of and research that we all are to the academic would to direct our to the research questions that to be with academic medical informatics. Academic medical informatics in such as medical for and of new technology for information the issue is whether these research questions be by who the of their time to that are best by and by information at own and the academic at has many and academic own medical the of in the of understanding the of members to be of our of the to the in the of to be our new the of to be in our do the of to be for to of we in any of is an service role that to be we are academic questions that to be The people who these two of are not the The for the two of are Stead that the of are all who out service within their and do not to the significant service that have of these of our field a research the was a different was no clinical in academic were no information systems that were available and it was the of who were on the academic to the of on the to with the clinical questions that were at The of the had to address significant service issues in their at the was no one to do is of the of these that is now a and that address the information technology of medical an with of us in would be to to The of the were their at a time the of systems was different from it is In the to the of has we are to the of to how to the of to with the of systems the that in of in the now of are and to There has a new of to with the systems that are now in that in are the to be the systems that we in the clinical of us who are in academic centers not to have with in the the on the affirmative that in the AMIA Fall Symposium is to have in the the of the clinical of The information systems available for in the medical centers are only The not have to on of us in to the will have a key role in that information systems that Warner for clinical will the more that information the of medical informatics research their the more will We are to debate the of and the role of medical informatics in the academic of us in medical informatics to with to the systems that we have and to the systems are used in our of on the have in our in that our a that is in the for or The for this is that we not to our in that our to the medical informatics of our is in medical informatics must with the of the it is much to frame the research questions in that our field to our that of us in are not suited to the of that Stead and Warner Slack would us to now that the field of medical informatics many of our are with the of such that they to have of how our with academic the medical informatics to a of such as the one at how on to research in the on the that information systems have had on and on the in with such How we the to by medical informatics who have in the of How the we do in the of clinical to the of people in the who knowledge and We in medical informatics have to to our but we have not in the of our we to who have both the and the time to that the we are and the we are to and to in medical informatics is it and significant and it the understanding of medical and medical is only as a that our we for people who in a medical at do it is for to a for a for an information that has a significant of it is with to the for to do the is role should academic medical informatics in of within the academic were a and on in the would it to How has the medical informatics research at in How many have that have a in or it is to that the we have has had on the that people have are out that we have had in our in our own but that the broad of the of that we have had in our that the research at is we have a at this and a research has have all but is that they are for order the people the the by order and the these have Mark and Warner are is a position. on personal and would be the to with their the of clinical information systems in this would be the to that of the systems in the will be much In rather that many of the systems we are to will not The with and is that their is not are as if is only and the of a of a and information service for do it all with a not a single that a and do and create an for to do and be a in the research or to many do one and the but be the and for and Warner are that put in of of information systems all We should only be that we would have or of these who a service create a and in an the only is people to do it if the of our medical centers believed in the of informatics as much as we and if we and Warner we would have an generation of technology of to the if this were the academic would not be in its their is in as a for the as any model. it is for a that is the of organizational in a that in the the who is for a that is to a the a different from the one as the for this is to that many service only at the of our in has raised as a of in informatics such are use the in are a and by medical informatics who are now in the but to their academic medical informatics that it is to do medical informatics that if to a the the and do This is not for the but it has its do not have to be at the with a service to it but must and with service would by that if with service and should not be these issues. should an such is the rather than the between research and service are a for but in the is the of these two different of Warner First of the that and much on research is most of us would that research in an academic is in the clinical the and that clinical in an academic as in an academic as would that a with an academic not be to do would argue that within the academic medical center we would the to have an academic To clinical from an academic in to a of the as as to the of medical In would to Mark Frisse and Mark Musen for is an position. Mark In the role of academic in medical we should to the all academic at the of this that academic within medical should not be but should be to the research by believed that the of who new medical to be by who to new rather than by role was to the or to In medical is an for academic their on rather than on academic are not by the of service the is an for a as Warner Slack The are have the time to address broad issues. The from the of a to the questions that to be next and in has a in medical informatics is that often our best people are service roles, the next is to that the field are the of the Academic medical centers in medical informatics who be on that who are not by the of order to were in medical many of the of clinical that were raised in this debate would We would have clinical not would have the to in in our academic medical our academic groups would create such that the would the to our their members of medical school we would be to the who have the in our and that academic We that the of academic in medical informatics should be education and is not to of these that or and systems for the clinical we of medical informatics with more more to our will have and our will be more both in academic centers and in the William This debate on two issues. The is the that to have an have to have one who it is not we are We are an team in people across roles. focus on of the a The issue is the people in academic informatics should have for that their has do not we the for a that a in the that is the only for an academic center to in informatics. at will that they any and they their have a team of and has informatics is a to to take for that or we will not have any role in the The statements of and the participants have us with much to the debate a or had the of the participants to a would be a from this This not the of both the as a and also the different we organize ourselves to do our This debate will in the if the debaters' our organizational focus and institutions the to organize their informatics these to their The of the debate to the participants for their and for the and of their positions. also the ACMI and for its in the of the session and the

This paper describes a medical application of modular neural networks (NNs) for temporal pattern recognition. In order to increase the reliability of prognostic indices for patients living with the acquired immunodeficiency syndrome (AIDS), survival prediction was performed in a system composed of modular NNS that classified cases according to death in a certain year of follow-up. The output of each NN module corresponded to the probability of survival in a given year. Inputs were the values of demographic, clinical and laboratory variables. The results of the modules were combined to produce survival curves for individuals. The NNs were trained by backpropagation and the results were evaluated in test sets of previously unseen cases. We showed that, for certain combinations of NN modules, the performance of the prognostic index, measured by the area under the receiver operating characteristic curve, was significantly improved (p 0.05). We also used calibration measurements to quantify the benefits of combining NN modules, and show why, when and how NNs should be combined for building prognostic models.

We have defined a knowledge-based framework for the creation of abstract, interval-based concepts from time-stamped clinical data, the knowledge-based temporal-abstraction (KBTA) method. The KBTA method decomposes its task into five subtasks; for each subtask we propose a formal solving mechanism. Our framework emphasizes explicit representation of knowledge required for abstraction of time-oriented clinical data, and facilitates its acquisition, maintenance, reuse and sharing. The RESUME system implements the KBTA method. We tested RESUME in several clinical-monitoring domains, including the domain of monitoring patients who have insulin-dependent diabetes. We acquired from a diabetes-therapy expert diabetes-therapy temporal-abstraction knowledge. Two diabetes-therapy experts (including the first one) created temporal abstractions from about 800 points of diabetic-patients' data. RESUME generated about 80% of the abstractions agreed by both experts; about 97% of the generated abstractions were valid. We discuss the advantages and limitations of the current architecture.

This paper describes how we applied the PROTÉGÉ-II architecture to build a knowledge-based system that configures elevators. The elevator-configuration task was solved originally with a system that employed the propose-and-revise problem-solving method (VT). A variant of this task, here named the Sisyphus-2 problem, is used by the knowledge-acquisition community for comparative studies. PROTÉGÉ-II is a knowledge-engineering environment that focuses on the use of reusable ontologies and problem-solving methods to generate task-specific knowledge-acquisition tools and executable problem solvers. The main goal of this paper is to describe in detail how we used PROTÉGÉ-II to model the elevator-configuration task. This description provides a starting point for comparison with other frameworks that use abstract problem-solving methods. Beginning with the textual description of the elevator-configuration task, we analysed the domain knowledge with respect to PROTÉGÉ-II’s main goal: to build domain-specific knowledge-acquisition tools. We used PROTÉGÉ-II’s suite of tools to construct a knowledge-based system, called ELVIS, that includes a reusable domain ontology, a knowledge-acquisition tool, and a propose-and-revise problem-solving method that is optimized to solve the elevator-configuration task. We entered domain-specific knowledge about elevator configuration into the knowledge base with the help of a task-specific knowledge-acquisition tool that PROTÉGÉ-II generated from the ontologies. After we constructed mapping relations to connect the knowledge base with the method’s code, the final executable problem solver solved the test case provided with the Sisyphus-2 material. We have found that the development of ELVIS has afforded a valuable test case for evaluating PROTÉGÉ-II’s suite of system-building tools. Only projects based on reasonably large problems, such as the Sisyphus-2 task, will allow us to improve the design of PROTÉGÉ-II and its ability to produce reusable components.

Provision of automated support for planning protocol-directed therapy requires a computer program to take as input clinical data stored in an electronic patient-record system and to generate as output recommendations for therapeutic interventions and laboratory testing that are defined by applicable protocols. This paper presents a synthesis of research carried out at Stanford University to model the therapy-planning task and to demonstrate a component-based architecture for building protocol-based decision-support systems. We have constructed general-purpose software components that (1) interpret abstract protocol specifications to construct appropriate patient-specific treatment plans; (2) infer from time-stamped patient data higher-level, interval-based, abstract concepts; (3) perform time-oriented queries on a time-oriented patient database; and (4) allow acquisition and maintenance of protocol knowledge in a manner that facilitates efficient processing both by humans and by computers. We have implemented these components in a computer system known as EON. Each of the components has been developed, evaluated, and reported independently. We have evaluated the integration of the components as a composite architecture by implementing T-HELPER, a computer-based patient-record system that uses EON to offer advice regarding the management of patients who are following clinical trial protocols for AIDS or HIV infection. A test of the reuse of the software components in a different clinical domain demonstrated rapid development of a prototype application to support protocol-based care of patients who have breast cancer.

Problem-solving methods for knowledge-based systems establish the behavior of such systems by defining the roles in which domain knowledge is used and the ordering of inferences. Developers can compose problem-solving methods that accomplish complex application tasks from primitive, reusable methods. The key steps in this development approach are task analysis, method selection (from a library), and method configuration. Protégé-ii is a knowledge-engineering environment that allows developers to select and configure problem-solving methods. In addition, Protégé-ii generates domain-specific knowledge-acquisition tools that domain specialists can use to create knowledge bases on which the methods may operate. The board-game method is a problem-solving method that defines control knowledge for a class of tasks that developers can model in a highly specific way. The method adopts a conceptual model of problem solving in which the solution space is construed as a “game board” on which the problem solver moves “playing pieces” according to prespecified rules. This familiar conceptual model simplifies the developer's cognitive demands when configuring the board-game method to support new application tasks. We compare configuration of the board-game method to that of a chronological-backtracking problem-solving method for the same application tasks (for example, towers of Hanoi and the Sisyphus room-assignment problem). We also examine how method designers can specialize problem-solving methods by making ontological commitments to certain classes of tasks. We exemplify this technique by specializing the chronological-backtracking method to the board-game method.

OBJECTIVE: A general framework for representation of clinical data that provides a declarative semantics of terms and that allows developers to define explicitly the relationships among both terms and combinations of terms. DESIGN: Use of conceptual graphs as a standard representation of logic and of an existing standardized vocabulary, the Systematized Nomenclature of Medicine (SNOMED International), for lexical elements. Concepts such as time, anatomy, and uncertainty must be modeled explicitly in a way that allows relation of these foundational concepts to surface-level clinical descriptions in a uniform manner. RESULTS: The proposed framework was used to model a simple radiology report, which included temporal references. CONCLUSION: Formal logic provides a framework for formalizing the representation of medical concepts. Actual implementations will be required to evaluate the practicality of this approach.

One of the most difficult aspects of modeling complex dilemmas in decision-analytic terms is composing a diagram of relevance relations from a set of domain concepts. Decision models in many domains, however, exhibit certain prototypical patterns that can guide the modeling process. Concepts can be classified according to semantic types that have characteristic positions and typical roles in an influence-diagram model. The authors have developed a graph-grammar production system that uses such inherent interrelationships among terms to facilitate the modeling of medical decisions. The authors' system also can examine a set of graph-grammar rules to establish whether the grammar satisfies a number of properties that they have determined to be important in the derivation of influence-diagram models. The authors' findings suggest that syntactic patterns can lead to automated construction of decision models in domains other than medicine.>

Building a problem solver and acquiring the knowledge needed to operate it are the two central goals of knowledge engineering. to achieve these goals, knowledge engineers construct models of the domain and of the task of interest. the various approaches used for modeling, however, have so far failed to define methods and techniques that can be applied across domains and tasks, and to produce models that can be reused in future applications. In this article, we propose that both of these objectives can be achieved by the use of building blocks called mechanisms. We examine the composition of mechanisms and also show how these mechanisms can be manipulated to construct problemsolving methods. We present PROTÉGÉ-II, a knowledge-acquisition shell that uses problem-solving methods to drive the modeling of tasks, the automatic generation of knowledge-acquisition tools, and the control flow of the problem solver. the modeling of tasks, within the context of PROTÉGÉ-II, is illustrated with two examples: one from the game domain and another from the medical-therapy domain. In addition, we introduce the conceptual basis for a library of mechanisms that serves as a repository of reusable knowledge components. © 1993 John Wiley & Sons, Inc.

The manner in which a knowledge-acquisition tool displays the contents of a knowledge base affects the way users interact with the system. Previous tools have incorporated semantics that allow knowledge to be edited in terms of either the structural representation of the knowledge or the problem-solving method in which that knowledge is ultimately used. A more effective paradigm may be to use the semantics of the application domain itself to govern access to an expert system's knowledge base. This approach has been explored in a program called OPAL, which allows medical specialists working alone to enter and review cancer treatment plans for use by an expert system called ONCOCIN. Knowledge-acquisition tools based on strong domain models should be useful in application areas whose structure is well understood and for which there is a need for repetitive knowledge entry.