The Integrated Microbial Genomes (IMG) system integrates microbial community aggregate genomes (metagenomes) with genomes from all domains of life. IMG provides tools for analyzing and reviewing the structural and functional annotations of metagenomes and genomes in a comparative context. At the core of the IMG system is a data warehouse that contains genome and metagenome datasets provided by scientific users, as well as public bacterial, archaeal, eukaryotic, and viral genomes from the US National Center for Biotechnology Information genomic archive and a rich set of engineered, environmental and host associated metagenomes. Genomes and metagenome datasets are processed using IMG's microbial genome and metagenome sequence data processing pipelines and then are integrated into the data warehouse using IMG's data integration toolkit. Microbial genome and metagenome application specific user interfaces provide access to different subsets of IMG's data and analysis toolkits. Genome and metagenome analysis is a gene centric iterative process that involves a sequence (composition) of data exploration and comparative analysis operations, with individual operations expected to have rapid response time.

Gene context analysis determines the function of genes by examining the conservation of chromosomal gene clusters and co-occurrence functional profiles across genomes. This is based on the observation that functionally related genes are often collocated on chromosomes as part of so called "gene cassettes", and relies on the identification of such cassettes across a statistically significant and phylogenetically diverse collection of genomes. Gene context analysis is an important part of a genomic data management system such as the Integrated Microbial Genomes (IMG) system, which has one of the largest public genome collections. As of January 2013, IMG contains 3.3 million gene cassettes across 8,000 genomes. A gene context analysis in IMG performs many millions of comparisons among the cassettes and their functions. Using a traditional relational database management system, these cassettes and their functional characteristics are represented by a correlation table of more than 2 billion rows along with a dozen auxiliary tables. This correlation table requires 16.5 hours to build and a typical query requires 5 to 10 minutes to answer.

MOTIVATION: A rapidly increasing number of microbial genomes are sequenced by organizations worldwide and are eventually included into various public genome data resources. The quality of the annotations depends largely on the original dataset providers, with erroneous or incomplete annotations often carried over into the public resources and difficult to correct. RESULTS: We have developed an Expert Review (ER) version of the Integrated Microbial Genomes (IMG) system, with the goal of supporting systematic and efficient revision of microbial genome annotations. IMG ER provides tools for the review and curation of annotations of both new and publicly available microbial genomes within IMG's rich integrated genome framework. New genome datasets are included into IMG ER prior to their public release either with their native annotations or with annotations generated by IMG ER's annotation pipeline. IMG ER tools allow addressing annotation problems detected with IMG's comparative analysis tools, such as genes missed by gene prediction pipelines or genes without an associated function. Over the past year, IMG ER was used for improving the annotations of about 150 microbial genomes.

BACKGROUND: Accurate taxonomy is best maintained if species are arranged as hierarchical groups in phylogenetic trees. This is especially important as trees grow larger as a consequence of a rapidly expanding sequence database. Hierarchical group names are typically manually assigned in trees, an approach that becomes unfeasible for very large topologies. RESULTS: We have developed an automated iterative procedure for delineating stable (monophyletic) hierarchical groups to large (or small) trees and naming those groups according to a set of sequentially applied rules. In addition, we have created an associated ungrouping tool for removing existing groups that do not meet user-defined criteria (such as monophyly). The procedure is implemented in a program called GRUNT (GRouping, Ungrouping, Naming Tool) and has been applied to the current release of the Greengenes (Hugenholtz) 16S rRNA gene taxonomy comprising more than 130,000 taxa. CONCLUSION: GRUNT will facilitate researchers requiring comprehensive hierarchical grouping of large tree topologies in, for example, database curation, microarray design and pangenome assignments. The application is available at the greengenes website 1.

In this paper we present a method for applying data warehouse and on-line analytical processing concepts to gene expression data management. This method has been employed in developing the data management system that is used to host Gene Logic's GeneExpress(R) database products.

Summary form only given. The Object-Protocol Model (OPM) data management tools provide facilities for rapid development, documentation, and flexible exploration of scientific databases. The tools are based on OPM, an object-oriented data model which is similar to the ODMG standard, but also supports extensions for modeling scientific data. Databases designed using OPM can be implemented using a variety of commercial relational DBMSs, using schema translation tools that generate complete DBMS database definitions from OPM schemas. Further, OPM schemas can be retrofitted on top of existing databases defined using a variety of notations, such as the relational data model or the ASN.1 data exchange format, using OPM retrofitting tools. Several archival molecular biology databases have been designed and implemented using the OPM tools, including the Genome Database (GDB) and the Protein Data Bank (PDB), while other scientific databases, such as the Genome Sequence Database (GSDB), have been retrofitted with semantically enhanced views using the OPM tools.

The Object-Protocol Model (OPM) data management tools provide facilities for rapid development, documentation, and flexible exploration of scientific databases. The tools are based on OPM, an object oriented data model which is similar to the ODMG standard, but also supports extensions for modeling scientific data (L.A. Chen and V.M. Markowitz, 1995). Databases designed using OPM can be implemented using a variety of commercial relational DBMSs, using schema translation tools that generate complete DBMS database definitions from OPM schemas (L.A. Chen and V.M. Markowitz, 1996). Further OPM schemas can be retrofitted on top of existing databases defined using a variety of notations, such as the relational data model or the ASN.1 data exchange format, using OPM retrofitting tools (L.A. Chen et al., 1997).

Scientific databases (ScDBs) are used to archive and retrieve data describing objects of scientific inquiry. Since these ScDBs must provide continuous and efficient access to large communities of scientists, they are often developed with reliable commercial relational database management systems (DBMSs) or file systems. However, relational DBMSs and flat files do not provide constructs for representing directly ScDB-specific objects and experimental procedures, and therefore they are often hard to develop, maintain, and explore. In this paper, we present a retrofitting tool for constructing and maintaining ScDB views using an object-oriented data model, and describe our experience with retrofitting ScDBs that have been originally developed using relational DBMSs and file systems. The retrofitting tool is part of a data management toolkit based on the Object-Protocol Model (OPM). The OPM toolkit provides facilities for developing databases defined using OPM and for querying and browsing...

We examine the main requirements for modeling scientific experiments and propose constructs that fulfil these requirements. We show that existing object-oriented and semantic data models do not provide such constructs. Experiment (protocol) and object constructs can be combined in order to provide seamless object and experiment modeling. We present an example of combining protocol and object constructs into a unified framework, the Object-Protocol Model (OPM), and briefly describe the implementation of an OPM interface on top of commercial relational database management systems (DBMSs).>

A query language called the Concise Object Query Language (COQL) is described. COQL is unique in its conciseness, in its support of inheritance, and in the capabilities it provides for defining application-specific structures. The COQL-to-SQL translation, its implementation on top of a commercial relational DBMS, and the ways in which COQL can be used for constructing application-specific views for scientific applications are discussed. The typical three-level architecture approach for supporting data management applications and previous work on the translation of extended entity-relationships schemas into relational database management system schemas are reviewed.>

A merging technique for relational schemas consisting of relation-schemes, key dependencies, referential integrity constraints, and null constraints is presented. The author examines the conditions required for using this technique with relational database management systems that provide different mechanisms for maintaining null and referential integrity constraints. For relational schemas developed using an extended entity-relationship (EER) oriented design methodology, it is shown that a relation-scheme can be used for representing multiple object-sets not only for the standard binary many-to-one relationship-set structure, but for more complex structures as well.>

A common approach to database design is to describe the structures and constraints of the database application in terms of a semantic data model, and then represent the resulting schema using the data model of a commercial database management system. Often, in practice, Extended Entity-Relationship (EER) schemas are translated into equivalent relational schemas. This translation involves different aspects: representing the EER schema using relational constructs, assigning names to relational attributes, normalization, and merging relations. Considering these aspects together, as is usually done in the design methodologies proposed in the literature, is confusing and leads to inaccurate results. We propose to treat separately these aspects and split the translation into four stages (modules) corresponding to the four aspects mentioned above. We define criteria for both evaluating the correctness of and characterizing the relationship between alternative relational representations of EER schemas.

Although closely related, functional and structural aspects of information systems are usually modeled independently of each other. The author proposes to combine functional and structural modeling into an integrated design methodology. He chooses for this purpose two popular and widely used models, the data-flow-oriented functional model and the extended entity-relationship (EER) data model. Since functional modeling generally precedes structural modeling in the design of information systems, the author examines how functional specifications can be represented using EER constructs. The methodology is based on using EER existence dependencies for representing the functional constraints implied by the process interactions.>

Relational schemas consisting of relation-schemes, key dependencies and key-based inclusion dependencies (referential integrity constraints) are considered. Schemas of this form are said to be entity-relationship (EER)-convertible if they can be associated with an EER schema. A procedure that determines whether a relational schema is EER-convertible is developed. A normal form is proposed for relational schemas representing EER object structures. For EER-convertible relational schemas, the corresponding normalization procedure is presented. The procedures can be used for analyzing the semantics of existing relational databases and for converting relational database schemas into object-oriented database schemas.>

Although the relational representation of Entity-Relationship (ER) structures gained extensive coverage, scarce attention has been paid to the issue of correctness for such representations. Several mappings have been proposed for the representation of both ER and extended ER (EER) structures by relational schemas. The informal nature of most of these proposals, however, does not allow a precise evaluation of their correctness, nor a comparison of the various mappings. We propose a canonical relational representation for EER structures and prove its correctness. We claim that a relational schema represents correctly an EER structure if it has equivalent information-capacity with the corresponding canonical representation.

Entity-relationship (ER) consistency is defined, and a complete set of incremental and reversible restructuring manipulations is proposed for ER-consistent relational schemas. It is shown how the schema restructuring manipulations that are proposed back both interactive and view-integration methodologies for schema design.>