Open data analysis platforms are being adopted to support collaboration in science and business. Studies suggest that analytic work in an enterprise occurs in a complex ecosystem of people, data, and software working in a coordinated manner. These studies also point to friction between the elements of this ecosystem that reduces user productivity and quality of work. LabBook is an open, social, and collaborative data analysis platform designed explicitly to reduce this friction and accelerate discovery. Its goal is to help users leverage each other's knowledge and experience to find the data, tools and collaborators they need to integrate, visualize, and analyze data. The key insight is to collect and use more metadata about all elements of the analytic ecosystem by means of an architecture and user experience that reduce the cost of contributing such metadata. We demonstrate how metadata can be exploited to improve the collaborative user experience and facilitate collaborative data integration and recommendations. We describe a specific use case and discuss several design issues concerning the capture, representation, querying and use of metadata.

One of the most well-studied problems in data mining is computing association rules from large transactional databases. Often, the rule collections extracted from existing data-mining methods can be far too large to be carefully examined and understood by the data analysts. In this paper, we address exactly this issue of overwhelmingly large rule collections by introducing and studying the following problem: Given a large collection R of association rules we want to pick a subset of them S ⊆ R that best represents the original collection R as well as the dataset from which R was extracted. We first quantify the notion of the goodness of a ruleset using two very simple and intuitive definitions. Based on these definitions we then formally define and study the corresponding optimization problems of picking the best ruleset S ⊆ R. We propose algorithms for solving these problems and present experiments to show that our algorithms work well for real datasets and lead to large reduction in the size of the original rule collection.

The XML Wrapper exploits the federated database capabilities of DB2 to enable SQL applications to query XML data from a variety of sources, including files and web services. The XML Wrapper models hierarchical XML documents as families of virtual relational tables in a federated schema, which can then be queried to extract information from the XML and combine it with data from other sources. To keep the wrapper simple and efficient, we take advantage of the existing capabilitiesof DB2 to the greatest extent possible. Our approach uses the DB2 optimizer’s ability to explore the space of join orders and join methods to find good execution plans for queries that include XML documents. We also leverage DB2’s view and rewrite mechanisms to allow a simple wrapper to support the full range of SQL queries an application might submit.

In a large modern enterprise, information is almost inevitably distributed among several database management systems. Despite considerable attention from the research community, relatively few commercial systems have attempted to address this issue. This paper describes new technology that enables clients of IBM's DB2 Universal Database to access the data and specialized computational capabilities of a wide range of non-relational data sources. This technology, based on the Garlic prototype developed at the Almaden Research Center, complements and extends DB2's existing ability to federate relational data sources.The paper focuses on three topics. Firstly, we show how the DB2 catalogs are used as an extensible repository for the metadata needed to access remotely-stored information. Secondly, we describe how the Garlic approach to query planning, in which source-specific modules and the federated server cooperate to develop an optimized execution plan, has been realized in DB2. Lastly, we describe how DB2's query execution engine has been extended to support queries and functions that are evaluated remotely.

Vast amounts of life sciences data today reside in specialized data sources, with specialized query processing capabilities. Data from one source must often be combined with data from other sources to give users the information they desire. Database middleware systems such as Garlic allow users to combine data from multiple sources in a single query. Garlic provides the user with a virtual database to which they can pose arbitrarily complex queries, though the actual data needed to answer the query may be stored in several different sources, and those sources may not even possess all the functionality needed to answer such a query themselves. The Garlic technology, as incorporated in IBM's DB2 product, forms the basis of the DiscoveryLink service offering for the life sciences industry. We describe the DiscoveryLink offering, focusing on two key contributions of Garlic, the wrapper architecture and the query optimizer, and illustrate how it can be used to integrate life sciences data from heterogeneous data sources.

With the profusion of text databases on the Internet, it is becoming increasingly hard to find the most useful databases for a given query. To attack this problem, several existing and proposed systems employ brokers to direct user queries, using a local database of summary information about the available databases. This summary information must effectively distinguish relevant databases and must be compact while allowing efficient access. We offer evidence that one broker, GlOSS , can be effective at locating databases of interest even in a system of hundreds of databased and can examine the performance of accessing the GlOSS summeries for two promising storage methods: the grid file and partitioned hashing. We show that both methods can be tuned to provide good performance for a particular workload (within a broad range of workloads), and we discuss the tradeoffs between the two data structures. As a side effect of our work, we show that grid files are more broadly applicable than previously thought; inparticular, we show that by varying the policies used to construct the grid file we can provide good performance for a wide range of workloads even when storing highly skewed data.

The goal of the Garlic [1] project is to build a multimedia information system capable of integrating data that resides in different database systems as well as in a variety of non-database data servers. This integration must be enabled while maintaining the independence of the data servers, and without creating copies of their data. "Multimedia" should be interpreted broadly to mean not only images, video, and audio, but also text and application specific data types (e.g., CAD drawings, medical objects, …). Since much of this data is naturally modeled by objects, Garlic provides an object-oriented schema to applications, interprets object queries, creates execution plans for sending pieces of queries to the appropriate data servers, and assembles query results for delivery back to the applications. A significant focus of the project is support for "intelligent" data servers, i.e., servers that provide media-specific indexing and query capabilities [2]. Database optimization technology is being extended to deal with heterogeneous collections of data servers so that efficient data access plans can be employed for multi-repository queries.A prototype of the Garlic system has been operational since January 1995. Queries are expressed in an SQL-like query language that has been extended to include object-oriented features such as reference-valued attributes and nested sets. In addition to a C++ API, Garlic supports a novel query/browser interface called PESTO [3]. This component of Garlic provides end users of the system with a friendly, graphical interface that supports interactive browsing, navigation, and querying of the contents of Garlic databases. Unlike existing interfaces to databases, PESTO allows users to move back and forth seamlessly between querying and browsing activities, using queries to identify interesting subsets of the database, browsing the subset, querying the content of a set-valued attribute of a particularly interesting object in the subset, and so on.

Rufus is an information system that models user data with objects taken from a class system. Due to the importance of coping with changes to the schema, Rufus has adopted the conformity-based model of Melampus. This model enables Rufus to cope with schema changes more easily than traditional class- and inheritance-based data models. The paper reviews the Melampus data model and describes how it was implemented in the Rufus system. The authors show how changes to the schema can be accommodated with minimum disruption. They also review design decisions that contributed to streamlined schema evolution and compare this approach with those proposed in the literature.>

Computer system users today are inundated with a flood of semi-structured information, such as documents, electronic mail, programs, and images. Today, this information is typically stored in filesystems that provide limited support for organizing, searching, and operating upon this data, all operations that are vital to the ability of users to effectively use this data. Database systems provide good function for organizing, searching, managing and writing applications on structured data. Current database systems are inappropriate for semi-structured information because moving the data into the database breaks all existing applications that use the data. The Rufus system attacks the problems of semi-structured information by using database function to help users manage semi-structured information without requiring that the user's information reside in the database.

This paper presents our experience with implementing atomicity in two systems: the QuickSilver distributed file system and the Starburst relational database manager. Each of these systems guarantees that certain collections of operations done on behalf of their clients execute atomically, despite process, machine, or network failures. In this paper we describe the atomic properties implemented by each system, present the algorithms and mechanisms used, examine the similarities and differences between the two systems, and give the rationale for different design decisions. We demonstrate that the support of atomicity with high performance requires a variety of techniques carefully chosen to balance the amount of data logged, the level of concurrency allowed, and the mutual consistency requirements of sets of objects, The main goal is to help others implement efficient systems that support atomicity.>

CACL is a protection scheme for objects that offers a simple and flexible model of protection and has an efficient, software-only implementation.The model, based on Access Control Lists (ACLs) integrated with the type system, allows owners to control who may invoke which methods on which objects, permits cooperation between mutually suspicious principals, allows ownership of objects to be transferred safely, prevents unwanted propagation of authority between principals, and allows changes to the authorization information to take effect on the next method invocation.The implementation, based on the integration of Capabilities with method dispatch, avoids the overhead of access checking in the majority of invocations, at the cost of space for extra dispatch vectors.CACL offers a viable mechanism for finegrained protection in an object-oriented database system.

DB2 TM , IMS, and Tandem TM systems. ARIES is applicable not only to database management systems but also to persistent object-oriented languages, recoverable file systems and transaction-based operating systems. ARIES has been implemented, to varying degrees, in IBM's OS/2 TM Extended Edition Database Manager, DB2, Workstation Data Save Facility/VM, Starburst and QuickSilver, and in the University of Wisconsin's EXODUS and Gamma database machine.

article Aspects: extending objects to support multiple, independent roles Share on Authors: Joel Richardson IBM Almaden Research Center, San Jose, California IBM Almaden Research Center, San Jose, CaliforniaView Profile , Peter Schwarz IBM Almaden Research Center, San Jose, California IBM Almaden Research Center, San Jose, CaliforniaView Profile Authors Info & Claims ACM SIGMOD RecordVolume 20Issue 2June 1991 pp 298–307https://doi.org/10.1145/119995.115839Online:01 April 1991Publication History 124citation521DownloadsMetricsTotal Citations124Total Downloads521Last 12 Months7Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

The TABS prototype is an experimental facility that provides operating system-level support for distributed transactions that operate on shared abstract types. The facility is designed to simplify the construction of highly available and reliable distributed applications. This paper describes the TABS system model, the TABS prototype's structure, and certain aspects of its operation. The paper concludes with a discussion of the status of the project and a preliminary evaluation.

The synchronization issues that arise when transaction facilities are extended for use with shared abstract data types are discussed.A formalism for specifying the concurrency properties of such types is developed, based on dependency relations that are defined in terms of an abstract type's operations.The formalism requires that the specification of an abstract type state whether or not cycles involving these relations should be allowed to form.Directories and two types of queues are specified using the technique, and the degree to which concurrency is restricted by type-specific properties is exemplified.How the specifications of types interact to determine the behavior of transactions is also discussed.A locking technique is described that permits implementations to make use of type-specific information to approach the limits of concurrency.