The Greater Plains Collaborative (GPC) is composed of 10 leading medical centers repurposing the research programs and informatics infrastructures developed through Clinical and Translational Science Award initiatives. Partners are the University of Kansas Medical Center, Children's Mercy Hospital, University of Iowa Healthcare, the University of Wisconsin-Madison, the Medical College of Wisconsin and Marshfield Clinic, the University of Minnesota Academic Health Center, the University of Nebraska Medical Center, the University of Texas Health Sciences Center at San Antonio, and the University of Texas Southwestern Medical Center. The GPC network brings together a diverse population of 10 million people across 1300 miles covering seven states with a combined area of 679 159 square miles. Using input from community members, breast cancer was selected as a focus for cohort building activities. In addition to a high-prevalence disorder, we also selected a rare disease, amyotrophic lateral sclerosis.

The increasing adoption of electronic health records (EHRs) and the corresponding interest in using these data for quality improvement and research have made it clear that the interpretation of narrative text contained in the records is a critical step. The biomedical literature is another important information source that can benefit from approaches requiring structuring of data contained in narrative text. For the first time, we dedicate an entire issue of JAMIA to biomedical natural language processing (NLP), a topic that has been among the most cited in this journal for the past few years. We start with a description of a contest to select the best performing algorithms for detection of temporal relationships in clinical documents (see page 806), followed by a general review of significance and brief description of commonly used methods to address this task (see page 814). Top performing approaches are featured in seven articles from five different countries—Canada (see page 843), China (see page 849), France (see page 820), Serbia (see page 859), and the US (see page 828, 836, 867).

Electronic medical record systems (EMRs) can be made more attractive to the clinicians if Clinical Decision Support Systems (CDSS) are integrated with them. However, CDSS have to be developed with integration in mind, such that they may be integrated not just with the local EMR but EMRs developed by others as well. Web Services Technology ameliorates the challenge of integration if the CDSS is well-designed but several other issues still need to be considered. The integration has to allow two-way data exchange between the CDSS and the EMR, which requires the EMR also to expose a set of interfaces. Further, the CDSS itself needs integration with services on which it depends for its functionality. In the semantic data capture initiative (SDCI) project, we integrated, Proteus (http://proteme.org), an open source, process-oriented clinical decision support system with Henry Ford Health System's EMR, Care Plus. The effort involved addressing some of these challenges and some that are unique to a system like Proteus.

This issue of JAMIA focuses on natural language processing (NLP) techniques for clinical-text information extraction. Several articles are offshoots of the yearly ‘Informatics for Integrating Biology and the Bedside’ (i2b2) (http://www.i2b2.org) NLP shared-task challenge, introduced by Uzuner et al (see page 552)1 and co-sponsored by the Veteran's Administration for the last 2 years. This shared task follows long-running challenge evaluations in other fields, such as the Message Understanding Conference (MUC) for information extraction,2 TREC3 for text information retrieval, and CASP4 for protein structure prediction. Shared tasks in the clinical domain are recent and include annual i2b2 Challenges that began in 2006, a challenge for multi-label classification of radiology reports sponsored by Cincinnati Children's Hospital in 2007,5 a 2011 Cincinnati Children's Hospital challenge on suicide notes,6 and the 2011 TREC information retrieval shared task involving retrieval of clinical cases from narrative records.7 Although NLP research in the clinical domain has been active since the 1960s, progress in the development of NLP applications for clinical text has been slow and lags behind progress in the general NLP domain. There are several barriers to NLP development in the clinical domain, and shared tasks like the i2b2/VA Challenge address some of these barriers. Nevertheless, many barriers remain and unless the community takes a more active role in developing novel approaches for addressing the barriers, advancement and innovation will continue to be slow. Historically, there have been substantial barriers to NLP development in the clinical domain. These barriers are not unique to the clinical domain: they also occur in the fields of software engineering and general NLP. Because of concerns regarding patient privacy and worry about revealing unfavorable institutional practices, hospitals and clinics have been extremely reluctant to allow access to clinical data for researchers from outside the associated institutions. The lack of reliable and inexpensive de-identification techniques for narrative reports has compounded the reluctance to share. Such restricted access to shared datasets has hindered collaboration and inhibited the ability to assess and adapt NLP technologies across institutions and among research groups. Several pioneering efforts5,8–11 have made clinical data available for sharing—we need more of these grass-roots efforts. Closely related but not completely conditional on lack of shared datasets is the deficiency of annotated clinical data for training NLP applications and benchmarking performance. The sublanguage of clinical reports often necessitates domain-specific development and training, and, as a consequence, NLP modules developed for general text typically do not perform as well on clinical narratives. We need increased coordination to create annotation sets that can be merged to produce larger training and evaluation sets. Without the ability to share data, the community has lacked incentives for developing common data models for manual and automatic annotations. The result is that annotated datasets are usually unique to the laboratory that generated them and thus remain small and that NLP modules that perform the same tasks cannot be substituted and compared without considerable translational effort. At present, the clinical NLP community is leveraging existing standards and conventions and working together to develop shared data models and to map annotations across information extraction applications. Adopting an existing NLP application or module is complicated—source code and documentation may be unavailable, and published descriptions may lack sufficient detail for reproducibility. Open source releases of clinical information extraction and retrieval systems have improved the opportunity to reproduce performance.12–15 Even with open source release, a tool may work less well in others' hands than in the hands of the original developers. Compounding the problem of reproducibility is the fact that proof-of-concept tools created in academic/research environments may not meet the highest software engineering quality, maintainability, scalability, or usability standards. And sometimes a tool may be over-fitted to a particular application, and modification to solve a similar problem may require wholesale changes. As Pedersen asserted,16 the NLP community needs to invest more in assisting others in applying and reproducing our results. In part due to previously listed barriers, collaboration within the clinical NLP community has been nominal. Development of NLP systems within the academic environment has centered around single institutions and single laboratories, and rather than building upon the foundations of previous work, the majority of clinical NLP systems developed over the last four decades have been reinvented as silos that are neither expanded nor applied outside of the individual laboratory. Other factors limiting collaboration include insufficient infrastructure for facilitating cooperation and the reality that collaboration is inherently inefficient. Nevertheless, as with the biomedical research community at large, a surge in progression beyond the last half century of research can only come through enhanced teamwork. A recent trend for teamwork across NLP research laboratories is evident in funded initiatives such as the VA's Consortium for Healthcare Informatics Research (CHIR)17 and in the ONC-funded SHARP Area 4 grant for Secondary Use of EHR data.18 Also, advances in open source development and conformity to common frameworks has led to recent advances in NLP allowing one team to extend work by another (eg, HiTEX12 built on GATE and cTAKES,14 ODIE,19 and Automated Retrieval Console (ARC) built on UIMA16). Although we are improving incrementally the predictive performance of clinical NLP tools, clinical NLP applications are seldom deployed in clinical, public health, or health services research settings. Currently, the perceived cost of applying NLP outweighs the perceived benefit. Deploying an NLP system typically requires a substantial amount of time from an expert NLP developer—normally, applications do not generalize and must be rebuilt, retrained, enhanced, and re-evaluated for each new task; the output of an NLP system typically requires extensive mapping to the specific problem being addressed; and the ability to aid a user in customizing the application is generally inadequate (see page 544).20 We need a shift of focus from accuracy in one task to generalizabiliy across many and from the production of papers as the sole output to production of usable software for medically relevant applications. We also need to understand where NLP tools fit into an overall user workflow so that the tools can be integrated into end-to-end applications for clinical, public health, and clinical research users. Shared tasks like the i2b2/VA Challenge address several of these barriers in part. Shared tasks provide annotated datasets to participants and sometimes to non-participants (i2b2 datasets are available to others a year after the Challenge). The i2b2 shared task is standardizing its corpus as much as possible—the same records are used from one year to the next with layers of annotation that build on each other, and common input/output specifications are applied every year. Shared tasks partially address the barrier of reproducibility by providing an evaluation opportunity that minimizes the risk of over-fitting: participants have time to train their systems in supervised fashion with an annotated training dataset, but then evaluation must be performed against a separate non-annotated dataset within a stringent time limit that prevents non-trivial system modifications. Although shared tasks are not designed for this purpose, the i2b2 Challenge has been the impetus for some new collaborations across independent research groups. Shared tasks have driven progress in related fields. For example, progress in speech understanding research was driven by a series of evaluations funded by DARPA from the late 1980s to the early 2000s.21 The research community was able to consistently drive down the error rate by a factor of two every 2 years, on successively more challenging tasks, moving from recognition of small-vocabulary read speech to automated transcription of broadcast news in multiple languages. Associated with this progress was the incorporation of speech recognition products into applications, from dictation to speech interfaces. Shared tasks provide value to the NLP community in several ways: Common evaluation metrics are developed. Annotated datasets are made available. Enticed by available annotated datasets, researchers in overlapping fields (both academic and corporate) participate in the tasks, bringing in new people and new approaches. Benchmarking evaluation on a shared dataset reveals the state-of-the-art performance for a given task. Students and post-docs receive excellent training opportunities. Preliminary results can be obtained by a new research group, which can potentially lead to funding opportunities. Pre-processed, standardized corpora with multiple layers of annotations on the same corpus pave the way for end-to-end evaluations in addition to evaluation on a single annotation layer. Conventions for standardizing annotations and input/output formats are developed, and despite other standardization efforts, shared task corpora often set de facto standards. In spite of the value of shared tasks, the tasks have several shortcomings: Participants come mainly from teams with funded projects that overlap with the shared task. For academic participants, a significant motivation is the opportunity to publish; however, there is sometimes limited value for the larger community in publications resulting from a shared task. Because development time is limited during shared tasks, participants often build on applications that already exist and apply methods already described in the literature. This can result in many similar approaches being applied to the same task. Although publishing the high-performing systems can be interesting, the resulting publications may not be novel and therefore may not improve the general body of knowledge. If a particular challenge task is repeated over time, there is a tendency for system approaches to converge on the approach that showed most success in the previous evaluation—evaluations repeated over time tend to reduce the diversity of approaches. Although shared tasks contribute to growth and progress, increased benefit to the community of clinical NLP developers and to potential users will require additional individual and community efforts that target existing barriers creatively. Driving progress in a way that will increase the impact of NLP in the realm of individual and population health will require creativity at both the grass-roots and the community levels. No single activity can tackle all barriers. In addition to encouraging variations on the development of shared tasks and their incentives, we would like to see new types of shared activities that foster the outcomes described below. In exchange for access to the costly annotated dataset, shared task participation could be contingent on depositing code in a shared repository or creating a web service for prospective users. In this model, the organizers could send test data to the participants' servers and the servers return the results for evaluation (see Leitner et al22 for a description of a metaserver used in evaluation of results from BioCreative II). The servers (and metaserver) could even persist, providing services to interested users beyond the initial shared task. Publication of computational methods in biomedical informatics journals like JAMIA could further encourage reproducibility of results through policies mandating simultaneous submission of code with a manuscript, as recommended by Pedersen.16 The clinical NLP community could independently accelerate reproducibility (and lead by example) by depositing code and developing web services in a common repository.23 This trend is occurring in settings restricted by affiliation, such as the VA VINCI framework24 (available only to VA researchers) and a cloud environment being hosted by the SHARP Area 4 grant18 (available to grant participants). The new National Center for Biomedical Computing iDASH25 is developing a similar cyber-infrastructure that will be restricted not by affiliation but by adherence to privacy policies and agreements required by data contributors. The National Library of Medicine is currently hosting a registry developed by the AMIA NLP working group called ORBIT for listing and pointing to biomedical informatics and NLP resources.26 In a shared task, dozens of research groups duplicate the same task independently. Although a variety of techniques can emerge for the same task, given the relatively short time frame allowed for development and training, the features and approaches applied in the challenge are often very similar. Whereas similarity of approaches reveals agreement among teams on the best approaches and sets the stage for collaboration, the competitive nature of shared tasks provides a disincentive to collaboration; the reward system for shared tasks is not at all dependent on the ability to collaborate across teams but is solely geared toward competition in which a single winner arises. The open source development community has found inherent rewards in collaborative development through supportive environments like GitHub. Perhaps we can learn from collaborative development communities who participate in hackathons27 and from the games industry in asking how a shared task can be designed so that collaboration is rewarded and becomes worthwhile, interesting, and attractive.28 Evaluation of a shared task is focused on accuracy, and existing challenges evaluate only predictive performance, not software engineering characteristics or usability. Imagine a shared task in which success is judged on usability of a system or direct portability of one technique to a new task or domain. Evaluating success of a system with this paradigm is inherently more complex, but we could learn from groupware evaluation, from the rich field of usability testing, and from incentivized competitions like those sponsored by the X-Prize Foundation.29 Because of the cost of creating annotated training data, shared tasks are often small scale, at least relative to real medical applications. We need new approaches to rapid adaptation of NLP systems to new applications, with less dependence on ‘deeply annotated’ data; such applications would present important opportunities for collaboration with the end user community, who might be motivated to provide domain expertise if they were likely to get a scalable, maintainable system out of the collaboration. Scalability will require more efficient techniques for manual annotation. And scalability will require an enriched ability to produce high quality software, which may necessitate better collaboration with industry30 and funding models that include support for operational development. The shared i2b2 evaluations have made a huge contribution to stimulating and vitalizing the field of clinical NLP; however, to ensure the transition into usable applications, the clinical NLP research community needs to address the critical issues of data access, development of shared infrastructure, and integration of software engineering methods to ensure the usability, maintainability, and availability of clinical NLP tools that are integrated into the workflow of real biomedical applications. This must be done in close collaboration with end users, software engineers, and clinical practitioners. We as a community need to think beyond the status quo of incremental improvement in the F score toward imaginative approaches that encourage collaboration, promote reproducibility, increase the scalability of NLP development, and provide value to end users. Authors are funded in part by U54HL108460, R01GM090187, SHARP ONC award 90TR0002, R01 CA127979, and U54LM008748. None. Not commissioned; internally peer reviewed.

OBJECTIVES: To provide an overview and tutorial of natural language processing (NLP) and modern NLP-system design. TARGET AUDIENCE: This tutorial targets the medical informatics generalist who has limited acquaintance with the principles behind NLP and/or limited knowledge of the current state of the art. SCOPE: We describe the historical evolution of NLP, and summarize common NLP sub-problems in this extensive field. We then provide a synopsis of selected highlights of medical NLP efforts. After providing a brief description of common machine-learning approaches that are being used for diverse NLP sub-problems, we discuss how modern NLP architectures are designed, with a summary of the Apache Foundation's Unstructured Information Management Architecture. We finally consider possible future directions for NLP, and reflect on the possible impact of IBM Watson on the medical field.

Case report forms (CRFs) are used for structured-data collection in clinical research studies. Existing CRF-related standards encompass structural features of forms and data items, content standards, and specifications for using terminologies. This paper reviews existing standards and discusses their current limitations. Because clinical research is highly protocol-specific, forms-development processes are more easily standardized than is CRF content. Tools that support retrieval and reuse of existing items will enable standards adoption in clinical research applications. Such tools will depend upon formal relationships between items and terminological standards. Future standards adoption will depend upon standardized approaches for bridging generic structural standards and domain-specific content standards. Clinical research informatics can help define tools requirements in terms of workflow support for research activities, reconcile the perspectives of varied clinical research stakeholders, and coordinate standards efforts toward interoperability across healthcare and research data collection.

The author discusses the challenges of pharmacovigilance using electronic medical record and claims data. Use of ICD-9 encoded data has low sensitivity for detection of adverse drug events (ADEs), because it requires that an ADE escalate to major-complaint level before it can be identified, and because clinical symptomatology is relatively under-represented in ICD-9. A more appropriate vocabulary for ADE identification, SNOMED CT, awaits wider deployment. The narrative-text record of progress notes can potentially be used for more sensitive ADE detection. More effective surveillance will require the ability to grade ADEs by severity. Finally, access to online drug information that includes both a reliable hierarchy of drug families as well as structured information on existing ADEs can improve the focus and predictive ability of surveillance efforts.

OBJECTIVE: To identify challenges in mapping internal International Classification of Disease, 9th edition, Clinical Modification (ICD-9-CM) encoded legacy data to Systematic Nomenclature of Medicine (SNOMED), using SNOMED-prescribed compositional approaches where appropriate, and to explore the mapping coverage provided by the US National Library of Medicine (NLM)'s SNOMED clinical core subset. DESIGN: This study selected ICD-CM codes that occurred at least 100 times in the organization's problem list or diagnosis data in 2008. After eliminating codes whose exact mappings were already available in UMLS, the remainder were mapped manually with software assistance. RESULTS: Of the 2194 codes, 784 (35.7%) required manual mapping. 435 of these represented concept types documented in SNOMED as deprecated: these included the qualifying phrases such as 'not elsewhere classified'. A third of the codes were composite, requiring multiple SNOMED code to map. Representing 45 composite concepts required introducing disjunction ('or') or set-difference ('without') operators, which are not currently defined in SNOMED. Only 47% of the concepts required for composition were present in the clinical core subset. Search of SNOMED for the correct concepts often required extensive application of knowledge of both English and medical synonymy. CONCLUSION: Strategies to deal with legacy ICD data must address the issue of codes created by non-taxonomist users. The NLM core subset possibly needs augmentation with concepts from certain SNOMED hierarchies, notably qualifiers, body structures, substances/products and organisms. Concept-matching software needs to utilize query expansion strategies, but these may be effective in production settings only if a large but non-redundant SNOMED subset that minimizes the proportion of extensively pre-coordinated concepts is also available.

Maintaining a large controlled biomedical vocabulary requires ensuring the content's internal consistency. This is done through rules, specified by the vocabulary's curators, which denote how the vocabulary's concepts should be defined. When individual organizations deploy such vocabularies, local concepts are typically added and linked to concepts in the main vocabulary: the process of maintaining and linking local content should follow the same rules. The operation of content-maintenance software can be facilitated by maintaining such rules in computable form. In this paper, we demonstrate how to implement computable rules for attribute usage in SNOMED CT using a table-driven approach where a given rule is expressed as one or more rows in a table and is consulted by generic code. This approach, which is tailored to database implementations, is computationally efficient and allows new attribute-definition rules to be created as data while needing minimal or no code modification.

OBJECTIVE: To devise an automated approach for integrating federated database information using database ontologies constructed from their extended metadata. BACKGROUND: One challenge of database federation is that the granularity of representation of equivalent data varies across systems. Dealing effectively with this problem is analogous to dealing with precoordinated vs. postcoordinated concepts in biomedical ontologies. MODEL DESCRIPTION: The authors describe an approach based on ontological metadata mapping rules defined with elements of a global vocabulary, which allows a query specified at one granularity level to fetch data, where possible, from databases within the federation that use different granularities. This is implemented in OntoMediator, a newly developed production component of our previously described Query Integrator System. OntoMediator's operation is illustrated with a query that accesses three geographically separate, interoperating databases. An example based on SNOMED also illustrates the applicability of high-level rules to support the enforcement of constraints that can prevent inappropriate curator or power-user actions. SUMMARY: A rule-based framework simplifies the design and maintenance of systems where categories of data must be mapped to each other, for the purpose of either cross-database query or for curation of the contents of compositional controlled vocabularies.

This article presents the latest developments in neuroscience information dissemination through the SenseLab suite of databases: NeuronDB, CellPropDB, ORDB, OdorDB, OdorMapDB, ModelDB and BrainPharm. These databases include information related to: (i) neuronal membrane properties and neuronal models, and (ii) genetics, genomics, proteomics and imaging studies of the olfactory system. We describe here: the new features for each database, the evolution of SenseLab's unifying database architecture and instances of SenseLab database interoperation with other neuroscience online resources.

In the current issue of JAMIA, Kawamoto and Lobach1 propose a software framework intended to facilitate widespread, effective, clinical decision support. The proposed framework embraces a service-oriented-architecture (SOA) approach. Service-oriented-architecture is a philosophy of design described as “the software equivalent of Lego bricks,”2 where a toolset of mix-and-match units (“services”), each performing a well-defined task, can reside on different machines (including geographically separated ones), ready to be used when needed. The most widespread implementations of SOA involve the use of Web services, where a given computational resource/service can be invoked by a remote machine via messages composed in XML and sent over HTTP, so that they can operate across firewalls. Think of a Web service, in its simplest form, as a subroutine that can be called over the Internet. Thanks to success stories such as Amazon.com,3 and at software giants such as SAP,4,5 the status of SOA in the business information technology (IT) domain has risen meteorically. The Kawamoto and Lobach paper1 reiterates the following potential benefits of SOA: Simpler software design and implementation, by decomposing complex problems into smaller, more manageable ones. Improved software reusability through enhanced reuse of existing IT resources. Improved adaptability to changing business requirements. Cost savings consequent to the above benefits. Some IT articles, however, present a “caveat emptor” skepticism about SOA.6,7 One must carefully inspect the claimed benefits of SOA closely, examine its potential drawbacks, and take a balanced approach to any proposed new SOA application while taking into consideration past lessons from business IT. In software design, simplification through problem decomposition into semi-independent units (subroutines, classes) is an important, incontrovertible principle. In SOA, where the units (services) lead a relatively autonomous existence (e.g., units are free to reside on physically separate hardware), decomposing a software solution into individual services makes sense only in two circumstances: When a particular service can provide independent value to the remote caller. When physical separation of services on multiple machines provides quantifiable scalability and performance benefits through hardware parallelism that more than offset the extra overhead of intermachine communication. Typically, the latter economies occur when the services are used mostly internally, and even then, achieving scalability may require upgrades to internal network infrastructure to minimize this overhead. When a user of Amazon.com asks for information on a book, for example, several dozen independent services compose individual parts of the returned Web page, such as product information, book rank, similar titles, and customer reviews. Newman7 points out that reuse of an existing resource requires that the resource was designed to be reusable. Designing, or redesigning, for reusability involves one or more of the following steps: Rigorously examining the assumptions implicit in an existing resource's design and determining which of these do not hold in a different circumstance. For example, consider the situation where a service's output must eventually be converted into human-readable text, as with Amazon's Web pages, or the recommendation system for preventive clinical procedures postulated in this issue's paper. One issue to be determined is whether the service should have a transnational reach, so that output is not restricted to the English language alone. Creating localizable software (i.e., software that can be adapted for different cultures without having to completely reprogram it) involves, among other things, isolating the textual elements that are to be shown to the user in “resource files” and referring to them symbolically. While software tools to facilitate localization are quite mature (e.g., Microsoft Visual Studio Team Suite®), reworking an existing software resource to make it localizable is inherently a painstaking and error-prone task. Identifying a general underlying problem amenable to creation of a general solution. Deciding to make something reusable often requires changing an organization's business model. Linquist2 cites ProCard, a credit-card service provider whose strengths resided in its software suite. ProCard determined that converting its suite into independently accessible services was worthwhile only if it transformed itself from a competitor of Visa and MasterCard to a provider of software services to its much larger rivals. Deciding whether the extra effort needed to re-engineer and/or generalize the solution is justified in the present situation, given constraints such as urgency and limited developer resources. The caveats here are that, as in great architecture and literature, simplicity is an elusive entity whose achievement is as much art as science: that is why it is so uncommon. Regarding reusability, Newman7 estimates that it generally takes at least three iterations to design a reusable software unit (class, library, service) well. Individuals capable of abstracting problems and devising elegant, simple, and general solutions tend to be both rare and expensive. It might be more accurate to state that SOA is a strategic, long-term objective rather than a tactical, short-term one. As with other system design approaches—e.g., metadata-driven software architectures—developers must invest a significant amount of effort in framework-building, which takes time, resources, and human expertise, before anticipated payoffs in reduced costs and greater adaptability are realized. For both Amazon and SAP, the time investment was at least four years. For a service that can provide independent value, the need to adhere to existing standards (when the service may be called from outside the organization, and must be interoperable with services designed by others) often necessitates extra effort that would not otherwise be required. Here the challenges concern the semantics of the data that must be passed to and from the service and how it must be represented, rather than the nature of the XML plumbing: modern software development environments mostly shield you from the latter and allow you to concentrate on the former. While the paper by Kawamoto and Lobach emphasizes the use of a standard, the HL7 Service Functional Model (SFM) Specification for Decision Support Service8 we note that this standard is in its early stages, with version 0.85 posted June 19, 2006 and version 1.0 posted July 23, 2006. In addition, we note that Dr. Kawamoto is a member of the HL7 SOA SIG, and was project leader on the HL7 Decision Support Service effort. Considerable experience will be needed to determine whether the specification is sufficiently comprehensive to meet its objectives and to what extent it will need to be revised. Amazon was fortunate in that they were the sole standards-setter for the family of services that they offer for external use. In an area where consensus is not yet present, however, the lead time required for consensus to be achieved must be factored in. In the medical domain, where HL7 version 3's status is still not official (for reasons that are not entirely technical but beyond the scope of this editorial), the reasons and process for implementing SOA must be similar to the ones in IT elsewhere. That is: The case for SOA must be made from a business perspective. The initial implementations of SOA must focus on internal use (where the services are used primarily over the organization's Intranet) so that one can worry less about nascent and immature standards and focus on solving specific problems. SOA must be understood to be a long-term, strategic approach that is not always applicable. In particular, as stated in Lindquist,2 if the reusability quotient of a given problem is low, a “one-off” non-SOA solution turns out to be more direct and faster to create. The paper by Kawamoto and Lobach1 mentions “service discovery” as one of the aspects of SOA. The idea of service discovery is that a service contains descriptive information about itself that would allow a remote automated software agent to determine whether a service exists on the Internet that would address a particular need. The HL7 specification discussed above provides for keywords that would assist the search for a particular service. While “semantic Web” researchers have made much of the potential of service discovery, this is currently far removed from reality. When carefully considered, service discovery is an extremely difficult problem to solve. If we regard the collection of biomedical services as a vast Internet-accessible subroutine library, determining whether a service is appropriate for a specific task would require considerable expert human intervention. Software developers who have to work with giant development frameworks such as Java® and Microsoft.NET®, which contain tens of thousands of subroutines, have to deal regularly with the same problem in a non–Web services setting. As often as not, browsing vendor documentation is insufficient, forcing the developer to use a Web search engine such as Google® and browse resources such as developer group forums and blogs. At present, the large Web service providers simply provide extensive documentation on each service, which also includes programming examples as well as case histories of successful use. The best service providers continually solicit feedback on the quality of their documentation in order to improve it. Clinicians, health care organizations, consumer groups, and informatics developers regard clinical decision support as a key method to address the inefficiencies and errors documented to occur in busy clinical practices.9,10 However, many health care organizations now see decision support as a selective means to provide better care to their patients in a manner that distinguishes them from their competitors. For the reasons listed above, SOA within a local environment may greatly facilitate such clinical decision support. However, there are potentially severe concerns if an organization “outsources” its decision support to potentially imperfect external agencies. The manner in which SOA is implemented, as noted by Kawamoto and Lobach, is as a series of “black boxes” that, given an input, produce an output. In general, SOA services are not “licensed practitioners,” so that legally, the patient's health care provider (clinician or institution) is responsible for overriding any erroneous advice provided by an SOA service. The latter circumstances may in and of themselves inhibit reliance on external clinical SOA services. If one were to ask a commercial software vendor, a pharmaceutical company, or a health care-related agency to provide a “medication dosing service” at a regional or national level, who would trust the service to provide uniformly correct answers? If a hypothetical SOA medication dosing service only took age into consideration, and not weight, it could not be used in pediatrics or for dosing certain medications, such as aminoglycoside antibiotics, in adults. Even if a dosing service took weight and age into consideration, but not gestational age or height, it still could not be used for neonatal applications or for situations in which doses are based on body surface area (e.g., chemotherapy). Dosing also depends on renal and hepatic function. Finally, dosing is often diagnosis-dependent. The same adult patient, at a given age, weight, height, and level of renal function, would require far higher doses of a “correct” beta lactam antibiotic when treating bacterial endocarditis in an inpatient setting than would be required to treat that patient's community-acquired pneumonia in an outpatient setting. How much information would a “medication dosing service” require as input in order to give proper advice in all settings, and how complex would the output have to be to cover a myriad of potential patient characteristics? In the best commercial drug databases on the market today, the algorithmic and data-structure sophistication for such a dosing service does not yet exist; the best that these systems do is reproduce the medication vendor's package insert text. Converting such an existing imperfect solution to a Web service would not be satisfactory. The logistics of responsibly implementing long-distance clinical SOA services may be overwhelming. The “black box” model for SOA services assumes relatively minimal input and output. Consider, however, the circumstance whereby the maintainers of a regional or national SOA clinical decision support service discover that the advice provided by the service has been faulty over the past 24 hours due to a “bug” introduced into the software (or the knowledge base) of the service. What mechanisms would exist that would enable the SOA provider to contact all care providers who had relied upon the service to provide advice for patients, and to then determine which patients' ordered regimens would need to be changed? To do so would require an “SOA transaction identifier” that at least goes back to identifying the institution requesting the service, and possibly even to the level of identifying the patient (at the SOA level). This raises the possibility whereby enough information about a patient must be transmitted to an SOA service to obtain advice that HIPAA privacy rules are violated. If SOA services are vended commercially (as might be the case in some future scenario involving the authors' commercialization efforts), one must consider how to charge the users of the services. Some of Amazon's services, e.g., product data (prices, images, customer reviews), are free, while other services are priced: e.g., the Historical Pricing service, which provides access to more than three years of Amazon's sales data for any item, has a fee of $249/month for up to 60,000 requests per month. The legal and ethical issues related to how such a commercial SOA decision support system might operate across state, and possibly national borders, have yet to be addressed in a definitive and thoughtful manner.

MOTIVATION: Packages that support the creation of pathway diagrams are limited by their inability to be readily extended to new classes of pathway-related data. RESULTS: VitaPad is a cross-platform application that enables users to create and modify biological pathway diagrams and incorporate microarray data with them. It improves on existing software in the following areas: (i) It can create diagrams dynamically through graph layout algorithms. (ii) It is open-source and uses an open XML format to store data, allowing for easy extension or integration with other tools. (iii) It features a cutting-edge user interface with intuitive controls, high-resolution graphics and fully customizable appearance. AVAILABILITY: http://bioinformatics.med.yale.edu CONTACTS: [email protected]; [email protected].

In highly functional metadata-driven software, the interrelationships within the metadata become complex, and maintenance becomes challenging. We describe an approach to metadata management that uses a knowledge-base subschema to store centralized information about metadata dependencies and use cases involving specific types of metadata modification. Our system borrows ideas from production-rule systems in that some of this information is a high-level specification that is interpreted and executed dynamically by a middleware engine. Our approach is implemented in TrialDB, a generic clinical study data management system. We review approaches that have been used for metadata management in other contexts and describe the features, capabilities, and limitations of our system.

Query Integrator System (QIS) is a database mediator framework intended to address robust data integration from continuously changing heterogeneous data sources in the biosciences. Currently in the advanced prototype stage, it is being used on a production basis to integrate data from neuroscience databases developed for the SenseLab project at Yale University with external neuroscience and genomics databases. The QIS framework uses standard technologies and is intended to be deployable by administrators with a moderate level of technological expertise: It comes with various tools, such as interfaces for the design of distributed queries. The QIS architecture is based on a set of distributed network-based servers, data source servers, integration servers, and ontology servers, that exchange metadata as well as mappings of both metadata and data elements to elements in an ontology. Metadata version difference determination coupled with decomposition of stored queries is used as the basis for partial query recovery when the schema of data sources alters.

OBJECTIVES: The authors designed and implemented a clinical data mart composed of an integrated information retrieval (IR) and relational database management system (RDBMS). DESIGN: Using commodity software, which supports interactive, attribute-centric text and relational searches, the mart houses 2.8 million documents that span a five-year period and supports basic IR features such as Boolean searches, stemming, and proximity and fuzzy searching. MEASUREMENTS: Results are relevance-ranked using either "total documents per patient" or "report type weighting." RESULTS: Non-curated medical text has a significant degree of malformation with respect to spelling and punctuation, which creates difficulties for text indexing and searching. Presently, the IR facilities of RDBMS packages lack the features necessary to handle such malformed text adequately. CONCLUSION: A robust IR+RDBMS system can be developed, but it requires integrating RDBMSs with third-party IR software. RDBMS vendors need to make their IR offerings more accessible to non-programmers.

The EAV/CR framework, designed for database support of rapidly evolving scientific domains, utilizes metadata to facilitate schema maintenance and automatic generation of Web-enabled browsing interfaces to the data. EAV/CR is used in SenseLab, a neuroscience database that is part of the national Human Brain Project. This report describes various enhancements to the framework. These include (1) the ability to create "portals" that present different subsets of the schema to users with a particular research focus, (2) a generic XML-based protocol to assist data extraction and population of the database by external agents, (3) a limited form of ad hoc data query, and (4) semantic descriptors for interclass relationships and links to controlled vocabularies such as the UMLS.

Clinical study data management systems (CSDMSs) have many similarities to clinical patient record systems (CPRSs) in their focus on recording clinical parameters. Requirements for ad hoc query interfaces for both systems would therefore appear to be highly similar. However, a clinical study is concerned primarily with collective responses of groups of subjects to standardized therapeutic interventions for the same underlying clinical condition. The parameters that are recorded in CSDMSs tend to be more diverse than those required for patient management in non-research settings, because of the greater emphasis on questionnaires for which responses to each question are recorded separately. The differences between CSDMSs and CPRSs are reflected in the metadata that support the respective systems' operation, and need to be reflected in the query interfaces. The authors describe major revisions of their previously described CSDMS ad hoc query interface to meet CSDMS needs more fully, as well as its porting to a Web-based platform.

The paper provides an overview of neuroinformatics research at Yale University being performed as part of the national Human Brain Project. This research is exploring the integration of multidisciplinary sensory data, using the olfactory system as a model domain. The neuroinformatics activities fall into three main areas: 1) building databases and related tools that support experimental olfactory research at Yale and can also serve as resources for the field as a whole, 2) using computer models (molecular models and neuronal models) to help understand data being collected experimentally and to help guide further laboratory experiments, 3) performing basic neuroinformatics research to develop new informatics technologies, including a flexible data model (EAV/CR, entity-attribute-value with classes and relationships) designed to facilitate the integration of diverse heterogeneous data within a single unifying framework.

OBJECTIVES: To test the hypothesis that most instances of negated concepts in dictated medical documents can be detected by a strategy that relies on tools developed for the parsing of formal (computer) languages-specifically, a lexical scanner ("lexer") that uses regular expressions to generate a finite state machine, and a parser that relies on a restricted subset of context-free grammars, known as LALR(1) grammars. METHODS: A diverse training set of 40 medical documents from a variety of specialties was manually inspected and used to develop a program (Negfinder) that contained rules to recognize a large set of negated patterns occurring in the text. Negfinder's lexer and parser were developed using tools normally used to generate programming language compilers. The input to Negfinder consisted of medical narrative that was preprocessed to recognize UMLS concepts: the text of a recognized concept had been replaced with a coded representation that included its UMLS concept ID. The program generated an index with one entry per instance of a concept in the document, where the presence or absence of negation of that concept was recorded. This information was used to mark up the text of each document by color-coding it to make it easier to inspect. The parser was then evaluated in two ways: 1) a test set of 60 documents (30 discharge summaries, 30 surgical notes) marked-up by Negfinder was inspected visually to quantify false-positive and false-negative results; and 2) a different test set of 10 documents was independently examined for negatives by a human observer and by Negfinder, and the results were compared. RESULTS: In the first evaluation using marked-up documents, 8,358 instances of UMLS concepts were detected in the 60 documents, of which 544 were negations detected by the program and verified by human observation (true-positive results, or TPs). Thirteen instances were wrongly flagged as negated (false-positive results, or FPs), and the program missed 27 instances of negation (false-negative results, or FNs), yielding a sensitivity of 95.3 percent and a specificity of 97.7 percent. In the second evaluation using independent negation detection, 1,869 concepts were detected in 10 documents, with 135 TPs, 12 FPs, and 6 FNs, yielding a sensitivity of 95.7 percent and a specificity of 91.8 percent. One of the words "no," "denies/denied," "not," or "without" was present in 92.5 percent of all negations. CONCLUSIONS: Negation of most concepts in medical narrative can be reliably detected by a simple strategy. The reliability of detection depends on several factors, the most important being the accuracy of concept matching.

OBJECTIVES: To explore the feasibility of using the National Library of Medicine's Unified Medical Language System (UMLS) Metathesaurus as the basis for a computational strategy to identify concepts in medical narrative text preparatory to indexing. To quantitatively evaluate this strategy in terms of true positives, false positives (spuriously identified concepts) and false negatives (concepts missed by the identification process). METHODS: Using the 1999 UMLS Metathesaurus, the authors processed a training set of 100 documents (50 discharge summaries, 50 surgical notes) with a concept-identification program, whose output was manually analyzed. They flagged concepts that were erroneously identified and added new concepts that were not identified by the program, recording the reason for failure in such cases. After several refinements to both their algorithm and the UMLS subset on which it operated, they deployed the program on a test set of 24 documents (12 of each kind). RESULTS: Of 8,745 matches in the training set, 7,227 (82.6 percent ) were true positives, whereas of 1,701 matches in the test set, 1, 298 (76.3 percent) were true positives. Matches other than true positive indicated potential problems in production-mode concept indexing. Examples of causes of problems were redundant concepts in the UMLS, homonyms, acronyms, abbreviations and elisions, concepts that were missing from the UMLS, proper names, and spelling errors. CONCLUSIONS: The error rate was too high for concept indexing to be the only production-mode means of preprocessing medical narrative. Considerable curation needs to be performed to define a UMLS subset that is suitable for concept matching.

BACKGROUND: The entity-attribute-value representation with classes and relationships (EAV/CR) provides a flexible and simple database schema to store heterogeneous biomedical data. In certain circumstances, however, the EAV/CR model is known to retrieve data less efficiently than conventionally based database schemas. OBJECTIVE: To perform a pilot study that systematically quantifies performance differences for database queries directed at real-world microbiology data modeled with EAV/CR and conventional representations, and to explore the relative merits of different EAV/CR query implementation strategies. METHODS: Clinical microbiology data obtained over a ten-year period were stored using both database models. Query execution times were compared for four clinically oriented attribute-centered and entity-centered queries operating under varying conditions of database size and system memory. The performance characteristics of three different EAV/CR query strategies were also examined. RESULTS: Performance was similar for entity-centered queries in the two database models. Performance in the EAV/CR model was approximately three to five times less efficient than its conventional counterpart for attribute-centered queries. The differences in query efficiency became slightly greater as database size increased, although they were reduced with the addition of system memory. The authors found that EAV/CR queries formulated using multiple, simple SQL statements executed in batch were more efficient than single, large SQL statements. CONCLUSION: This paper describes a pilot project to explore issues in and compare query performance for EAV/CR and conventional database representations. Although attribute-centered queries were less efficient in the EAV/CR model, these inefficiencies may be addressable, at least in part, by the use of more powerful hardware or more memory, or both.

The task of creating and maintaining a front end to a large institutional entity-attribute-value (EAV) database can be cumbersome when using traditional client-server technology. Switching to Web technology as a delivery vehicle solves some of these problems but introduces others. In particular, Web development environments tend to be primitive, and many features that client-server developers take for granted are missing. WebEAV is a generic framework for Web development that is intended to streamline the process of Web application development for databases having a significant EAV component. It also addresses some challenging user interface issues that arise when any complex system is created. The authors describe the architecture of WebEAV and provide an overview of its features with suitable examples.

OBJECTIVE: To query a clinical data repository (CDR) for answers to clinical questions to determine whether different types of fields (coded and free text) would yield confirmatory, complementary, or conflicting information and to discuss the issues involved in producing the discrepancies between the fields. METHODS: The appropriate data fields in a subset of a CDR (5,135 patient records) were searched for the answers to three questions related to surgical procedures. Each search included at least one coded data field and at least one free-text field. The identified free-text records were then searched manually to ensure correct interpretation. The fields were then compared to determine whether they agreed with each other, were supportive of each other, contained no entry (absence of data), or were contradictory. RESULTS: The degree of concordance varied greatly according to the field and the question asked. Some fields were not granular enough to answer the question. The free-text fields often gave an answer that was not definitive. Absence of data was most logically interpreted in some cases as lack of completion of data and in others as a negative answer. Even with a question as specific as which side a hernia was on, contradictory data were found in 5 to 8 percent of the records. CONCLUSIONS: Using the data in the CDR to answer clinical questions can yield significantly disparate results depending on the question and which data fields are searched. A database cannot just be queried in automated fashion and the results reported. Both coded and textual fields must be searched to obtain the fullest assessment. This can be expected to result in information that may be confirmatory, complementary, or conflicting. To yield the most accurate information possible, final answers to questions require human judgment and may require the gathering of additional information.

Entity-attribute-value (EAV) representation is a means of organizing highly heterogeneous data using a relatively simple physical database schema. EAV representation is widely used in the medical domain, most notably in the storage of data related to clinical patient records. Its potential strengths suggest its use in other biomedical areas, in particular research databases whose schemas are complex as well as constantly changing to reflect evolving knowledge in rapidly advancing scientific domains. When deployed for such purposes, the basic EAV representation needs to be augmented significantly to handle the modeling of complex objects (classes) as well as to manage interobject relationships. The authors refer to their modification of the basic EAV paradigm as EAV/CR (EAV with classes and relationships). They describe EAV/CR representation with examples from two biomedical databases that use it.

MOTIVATION: Molecular biology databases have been proliferating rapidly. Their heterogeneity and complexity pose a great challenge to efforts in database interoperation. To minimize the efforts of interoperating heterogeneous databases, it is useful to develop a system that lets a user of a particular genomic database access another related database as if the latter is structurally similar to the former. RESULTS: We extend a structurally simple model-the entity-attribute-value (EAV) model-to describe uniformly metadata relating to individual databases. Such metadata, which are necessary for performing database comparisons, include descriptions of primitive database objects (including entities, attributes, domain values and entity relationships) and specification of correspondences among the database objects. We show how to decompose SQL queries and map them from one database to another based on the EAV representation of the basic database objects. A prototype system is implemented to demonstrate query interoperation between two chromosome map databases. AVAILABILITY: Freely available (Cold Fusion source code and an Access database containing the mapping knowledge) upon request from the author. CONTACT: [email protected]

MOTIVATION: When two or more genomic maps of a chromosomal region are available, it is useful to be able to synthesize them to create a merged map. RESULTS: We show that map merging is an exploratory process because there are multiple ways to combine data based upon what the user wishes to focus on, and upon which particular data subset emphasis is desired. We describe Mapmerge, a program for merging two genomic maps, discuss its limitations, and illustrate an example of its use. AVAILABILITY: Freely available (ANSI C source code, a Make file, test data files, documentation) on request from the author. CONTACT: [email protected]

Entity--attribute--value (EAV) tables form the major component of several mainstream electronic patient record systems (EPRSs). Such systems have been optimized for real-time retrieval of individual patient data. Data warehousing, on the other hand, involves cross-patient data retrieval based on values of patient attributes, with a focus on ad hoc query. Attribute-centric query is inherently more difficult when data are stored in EAV form than when they are stored conventionally. The authors illustrate their approach to the attribute-centric query problem with ACT/DB, a database for managing clinical trials data. This approach is based on metadata supporting a query front end that essentially hides the EAV/non-EAV nature of individual attributes from the user. The authors' work does not close the query problem, and they identify several complex subproblems that are still to be solved.

ACT/DB is a client-server database application for storing clinical trials and outcomes data, which is currently undergoing initial pilot use. It stores most of its data in entity-attribute-value form. Such data are segregated according to data type to allow indexing by value when possible, and binary large object data are managed in the same way as other data. ACT/DB lets an investigator design a study rapidly by defining the parameters (or attributes) that are to be gathered, as well as their logical grouping for purposes of display and data entry. ACT/DB generates customizable data entry. The data can be viewed through several standard reports as well as exported as text to external analysis programs. ACT/DB is designed to encourage reuse of parameters across multiple studies and has facilities for dictionary search and maintenance. It uses a Microsoft Access client running on Windows 95 machines, which communicates with an Oracle server running on a UNIX platform. ACT/DB is being used to manage the data for seven studies in its initial deployment.

MOTIVATION: When inspecting two maps of a chromosome or chromosomal region derived by similar or different methodologies, a computer-generated description of their differences is valuable, both to guide laboratory research as well as to assist version control. Our program, Mapdiff, can be used to compare two independently derived maps, or two revisions of the same map. Its usefulness increases in proportion to the percentage of shared objects between the two maps. Mapdiff uses a greedy algorithm to determine differences between shared objects. RESULTS: We illustrate Mapdiff's use in comparing the publicly available STS-content and radiation hybrid maps of human chromosome 12. AVAILABILITY: Freely available, (source, executables for Windows/DOS and Sun Solaris, documentation) on request from the author.

Research groups within the Human Brain Project are developing technologies to help organize and make accessible the vast quantities of information being accumulated in the neurosciences. The goal of this work is to provide systems that enable this complex information from many diverse sources to be synthesized into a coherent theory of nervous system function. Our initial approach to this problem has been to create several small databases. While addressing the issues of each individual database, we are also considering how each might be incorporated into an integrated cluster of databases. In this paper, we describe a pilot project in which we construct a database of computational models of neuronal function. This database allows models to be created and run and their results reviewed through a World Wide Web interface. Because models encapsulate knowledge in a formal manner about how neuronal systems function, we also discuss how this database forms a natural center for our initial attempts at creating a cluster of related databases. General issues of database development in the context of the Web are also discussed.

This paper describes an approach that provides Internet-based support for a genome center to map human chromosome 12, as a collaboration between laboratories at the Albert Einstein College of Medicine in Bronx, New York, and the Yale University School of Medicine in New Haven, Connecticut. Informatics is well established as an important enabling technology within the genome mapping community. The goal of this paper is to use the chromosome 12 project as a case study to introduce a medical informatics audience to certain issues involved in genome informatics and in the Internet-based support of collaborative bioscience research. Central to the approach described is a shared database (DB/12) with Macintosh clients in the participating laboratories running the 4th Dimension database program as a user-friendly front end, and a Sun SPARCstation-2 server running Sybase. The central component of the database stores information about yeast artificial chromosomes (YACs), each containing a segment of human DNA from chromosome 12 to which genome markers have been mapped, such that an overlapping set of YACs (called a "contig") can be identified, along with an ordering of the markers. The approach also includes 1) a map assembly tool developed to help biologists interpret their data, proposing a ranked set of candidate maps, 2) the integration of DB/12 with external databases and tools, and 3) the dissemination of the results. This paper discusses several of the lessons learned that apply to many other areas of bioscience, and the potential role for the field of medical informatics in helping to provide such support.

Prakash M. Nadkarni, Stephen T. Reeders, Jing Zhou; A modification of the ALIGN program to assist visual interpretation of mutated sequences, Bioinformatic

It is widely accepted that parallel computers, which have the ability to execute different parts of a program simultaneously, will offer dramatic speed-up for many time-consuming biological computations. The paper describes how the use of the machine-independent parallel programming language, Linda, allows parallel programs to run on an institution's network of workstations. In this way, an institution can harness existing hardware, which is often either idle or vastly underutilized, as a powerful 'parallel machine' with supercomputing capability. The paper illustrates this very general paradigm by describing the use of Linda to parallelize three widely used programs for genetic linkage analysis, a mathematical technique used in gene mapping. The paper then discusses a number of technical, administrative and social issues that arise when creating such a computational resource.

A platform program that performs biological sequence comparison provides a case study to compare the relative advantages of a machine-independent approach to parallel computation versus a machine-specific approach. The program consists of two routines: (i) PSCANLIB, which compares a single biological sequence against a database of sequences, and (ii) PCOMPLIB, which compares a database of sequences against another database of sequences, or against itself. The program was first parallelized to run on the Intel Hypercube parallel computer using native Hypercube commands to coordinate the parallel computation. The parallelization logic of the program was then translated into a machine-independent parallel programming language, Linda. These two approaches to parallelization are contrasted in terms of: (i) the expressive power of the logic that coordinates the parallel computation, (ii) the portability of the machine-independent version to other parallel machines and (iii) the relative efficiency of the two versions of the program. In the benchmark tests reported, the benefits of the machine-independent approach were achieved with only a modest sacrifice in efficiency.

We have parallelized the FASTA algorithm for biological sequence comparison using Linda, a machine-independent parallel programming language. The resulting parallel program runs on a variety of different parallel machines. A straight-forward parallelization strategy works well if the amount of computation to be done is relatively large. When the amount of computation is reduced, however, disk I/O becomes a bottleneck which may prevent additional speed-up as the number of processors is increased. The paper describes the parallelization of FASTA, and uses FASTA to illustrate the I/O bottleneck problem that may arise when performing parallel database search with a fast sequence comparison algorithm. The paper also describes several program design strategies that can help with this problem. The paper discusses how this bottleneck is an example of a general problem that may occur when parallelizing, or otherwise speeding up, a time-consuming computation.