Database systems use static analysis to determine upfront which data is needed for answering a query and use indexes and other physical design techniques to speed-up access to that data. However, for important classes of queries, e.g., HAVING and top-k queries, it is impossible to determine up-front what data is relevant. To overcome this limitation, we develop provenance-based data skipping (PBDS), a novel approach that generates provenance sketches to concisely encode what data is relevant for a query. Once a provenance sketch has been captured it is used to speed up subsequent queries. PBDS can exploit physical design artifacts such as indexes and zone maps.

The variety of data is one of the important issues in the era of Big Data. The data are naturally organized in different formats and models, including structured data, semi-structured data, and unstructured data. Prior research has envisioned an approach to abstract multi-model data with a schema category and an instance category by using category theory. In this paper, we demonstrate a system, called MultiCategory, which processes multi-model queries based on category theory and functional programming. This demo is centered around four main scenarios to show a tangible system. First, we show how to build a schema category and an instance category by loading different models of data, including relational, XML, key-value, and graph data. Second, we show a few examples of query processing by using the functional programming language Haskell. Third, we demo the flexible outputs with different models of data for the same input query. Fourth, to better understand the category theoretical structure behind the queries, we offer a variety of graphical hooks to explore and visualize queries as graphs with respect to the schema category, as well as the query processing procedure with Haskell.

A well-established technique for capturing database provenance as annotations on data is to instrument queries to propagate such annotations. However, even sophisticated query optimizers often fail to produce efficient execution plans for instrumented queries. We develop provenance-aware optimization techniques to address this problem. Specifically, we study algebraic equivalences targeted at instrumented queries and alternative ways of instrumenting queries for provenance capture. Furthermore, we present an extensible heuristic and cost-based optimization framework utilizing these optimizations. Our experiments confirm that these optimizations are highly effective, improving performance by several orders of magnitude for diverse provenance tasks.

Database provenance explains how results are derived by queries. However, many use cases such as auditing and debugging of transactions require understanding of how the current state of a database was derived by a transactional history. We present MV-semirings, a provenance model for queries and transactional histories that supports two common multi-version concurrency control protocols: snapshot isolation (SI) and read committed snapshot isolation (RC-SI). Furthermore, we introduce an approach for retroactively capturing such provenance using reenactment, a novel technique for replaying a transactional history with provenance capture. Reenactment exploits the time travel and audit logging capabilities of modern DBMS to replay parts of a transactional history using queries. Importantly, our technique requires no changes to the transactional workload or underlying DBMS and results in only moderate runtime overhead for transactions. We have implemented our approach on top of a commercial DBMS and our experiments confirm that by applying novel optimizations we can efficiently capture provenance for complex transactions over large data sets.

Data provenance is essential for debugging query results, auditing data in cloud environments, and explaining outputs of Big Data analytics. A well-established technique is to represent provenance as annotations on data and to instrument queries to propagate these annotations to produce results annotated with provenance. However, even sophisticated optimizers are often incapable of producing efficient execution plans for instrumented queries, because of their inherent complexity and unusual structure. Thus, while instrumentation enables provenance support for databases without requiring any modification to the DBMS, the performance of this approach is far from optimal. In this work, we develop provenancespecific optimizations to address this problem. Specifically, we introduce algebraic equivalences targeted at instrumented queries and discuss alternative, equivalent ways of instrumenting a query for provenance capture. Furthermore, we present an extensible heuristic and cost-based optimization (CBO) framework that governs the application of these optimizations and implement this framework in our GProM provenance system. Our CBO is agnostic to the plan space shape, uses a DBMS for cost estimation, and enables retrofitting of optimization choices into existing code by adding a few LOC. Our experiments confirm that these optimizations are highly effective, often improving performance by several orders of magnitude for diverse provenance tasks.

Debugging transactions and understanding their execution are of immense importance for developing OLAP applications, to trace causes of errors in production systems, and to audit the operations of a database. However, debugging transactions is hard for several reasons: 1) after the execution of a transaction, its input is no longer available for debugging, 2) internal states of a transaction are typically not accessible, and 3) the execution of a transaction may be affected by concurrently running transactions. We present a debugger for transactions that enables non-invasive, postmortem debugging of transactions with provenance tracking and supports what-if scenarios (changes to transaction code or data). Using reenactment , a declarative replay technique we have developed, a transaction is replayed over the state of the DB seen by its original execution including all its interactions with concurrently executed transactions from the history. Importantly, our approach uses the temporal database and audit logging capabilities available in many DBMS and does not require any modifications to the underlying database system nor transactional workload.

Provenance for transactional updates is critical for many applications such as auditing and debugging of transactions. Recently, we have introduced MV-semirings, an extension of the semiring provenance model that supports updates and transactions. Furthermore, we have proposed reenactment, a declarative form of replay with provenance capture, as an efficient and non-invasive method for computing this type of provenance. However, this approach is limited to the snapshot isolation (SI) concurrency control protocol while many real world applications apply the read committed version of snapshot isolation (RC-SI) to improve performance at the cost of consistency. We present non trivial extensions of the model and reenactment approach to be able to compute provenance of RC-SI transactions efficiently. In addition, we develop techniques for applying reenactment across multiple RC-SI transactions. Our experiments demonstrate that our implementation in the GProM system supports efficient re-construction and querying of provenance.

Big Data applications need a situation aware computing model to manage data, knowledge, and processes in an ever increasing amount, complexity, and speed while reacting as efficiently and timely as possible to any evolving situation. We introduce a Knowledge Intensive Data-processing System (KIDS) that empowers Big Data applications to support situation awareness. Frameworks such as Apache Hadoop YARN can be leveraged for repeated and near real-time execution of knowledge intensive applications. KIDS bridges the gap between the world of low-value data and the world of high-value information and knowledge, which are best handled by state of the art databases. These databases provide a host of much needed functions such as multi-temporality, flashback, provenance, and registered queries. With KIDS model, Big Data applications are well structured and can evolve perpetually.

In a traditional database system, the transaction management protocols and mechanisms are constrained by the fundamental properties of atomicity, consistency, isolation, and durability (ACID). A transaction management system with strict ACID properties typically employs read and write locks, held for the duration of the transaction, to protect its uncommitted data from being seen and modified by some other transaction. While this approach is effective for applications involving short execution times and relatively small number of concurrent operations, it is too restrictive for applications that involve reactive, long-lived, and complex transactions. The common denominator of such applications is the need for transactions to read and possibly modify uncommitted data values [1] and for the database system to still retain the ability to abort a transaction and the ability to recover from failures. This paper proposes a Business Transaction framework that allows long lasting, discontinuous, and resumable transactions to perform shared updates to common data by holding semantic locks on the modified rows. Under this framework, basic SQL data types are made compensation-aware by associating domain-specific shared update semantics with them. These semantics ensure that each data modification operation is compatible with other uncommitted activity on the same data and that the operation can be undone, if needed, without resorting to cascading aborts. This paper describes the key concepts and presents our approach for supporting shared updates in Oracle RDBMS.

This tutorial deals with applications that help systems and individuals respond to critical conditions in their environments. The identification of critical conditions requires correlating vast amounts of data within and outside an enterprise. Conditions that signal opportunities or threats are defined by complex patterns of data over time, space and other attributes. Systems and individuals have models (expectations) of behaviors of their environments, and applications notify them when reality - as determined by measurements and estimates - deviate from their expectations. Components of event systems are also sent information to validate their current models and when specific responses are required. Valuable information is that which supports or contradicts current expectations or that which requires an action on the part of the receiver. A major problem today is information overload; this problem can be solved by identifying what information is critical, complementing existing pull technology with sophisticated push technology, and filtering out non-critical data.

The interaction with the Web has historically evolved from static bookmarks to dynamic searches to the current usage of active notification mechanisms based on popular protocols like RSS or Atom. In the same time a large volume of important source data is still contained in relational databases. The talk will analyze the way the Oracle database participates to the activation of the data and opening the state changes in a standard and secure way for easy integrating with the rest of the push based Web protocols. We will study the declarative specification of RSS feeds generated based on the state changes detected in the data stored in the Oracle database. On the opposite, external RSS feeds can be injected to the database and processed declaratively in conjunction with the rest of the data. Most of the technical pieces required for such a solution are already supported by the database engine (e.g. declarative XML processing, state change notifications, queues, crawlers, continuous queries), effectively turning the database into a declarative XML hub. The advantages of using database solutions for such problems in an enterprise context are security, scalability and reliability.

In this paper, we examine how active database technology developed over the past few years has been put to use to solve real world problems. We note how the technology had to be extended beyond the feature set originally identified in early research to meet these real-world needs, and discuss why this technology was best suited to solving these problems.

Database technology has done an excellent job of managing data. SQL92/99 and XML are generally considered to be powerful building blocks; these building blocks are complemented by support for Text, Images, Audio, Video, Spatial, Expressions, and other complex data structures. Database technology can also transparently manage access to data in other (remote) databases, in file systems, and in applications. Furthermore, database technology has achieved impressive operational characteristics with respect to, e.g., performance, scalability, reliability, component and site tolerance, and security.

The support for the expression data type in a relational database system allows storing of conditional expressions as data in database tables and evaluating them using SQL queries. In the context of this new capability, expressions can be interpreted as descriptions, queries, and filters, and this significantly broadens the use of a relational database system to support new types of applications. The paper presents an overview of the expression data type, relates expressions to descriptions, queries, and filters, considers applications pertaining to information distribution, demand analysis, and task assignment, and shows how these applications can be easily supported with improved functionality.

Over the last couple of years application integration has taken a central position in the business world. Application integration deals with integrating computing environments within and between companies and depends on connectivity provided by the intranet and Internet respectively. Application integration is typically referred to as EAI (e-business application integration). The article sketches first the evolution of business computing and EAI. The major elements of a modern EAI technology, are the focus of the discussion, with special attention to Web based application integration. Finally, the article points to some interesting research topics.

There is an ever growing interest in messaging and queuing technology; one needs only to look at the many new products that have been made available within the last few years. Oracle's approach is unique since it represents the only product that offers messaging and queuing fully integrated into a database server, Oracle8/sup TM/.

article Free Access Share on Report on the international workshop on high-performance transaction systems Author: Dieter Gawlick Amdahl Corporation, Sunnyvale, CA Amdahl Corporation, Sunnyvale, CAView Profile Authors Info & Claims ACM Transactions on Database SystemsVolume 11Issue 4Dec. 1986 pp 375–377https://doi.org/10.1145/7239.17346Online:01 December 1986Publication History 0citation246DownloadsMetricsTotal Citations0Total Downloads246Last 12 Months4Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF