The initial DB Cracking algorithm has spawned variations, offering distinct techniques and advantages concerning robustness and convergence. However, it is essential to consider the selection of the optimal list of attributes for indexing based on workloads and cost-based refinement of the indices. We propose a DB Cracking-based index management conceptualized as a two-player game to address these requirements. While leveraging the advantages of the underlying DB Cracking method, this model adapts index attribute selection and the degree of index refinement to any query workload. Furthermore, we introduce a new variant, Statistical DB Cracking, to complement the two-player model. Notably, an extensive experimental study validates the superiority of our model in terms of robustness and cost-effectiveness for index management compared to the direct DB Cracking algorithms.

Background: Distributed data-intensive systems are increasingly designed to be only eventually consistent. Persistent data is no longer processed with serialized and transactional access, exposing applications to a range of potential concurrency anomalies that need to be handled by the application itself. Controlling concurrent data access in monolithic systems is already challenging, but the problem is exacerbated in distributed systems. To make it worse, only little systematic engineering guidance is provided by the software architecture community regarding this issue. Aims: In this paper, we report on our study of the effectiveness and applicability of the novel design guidelines we are proposing in this regard. Method: We used action research and conducted it in the context of the software architecture design process of a multi-site platform development project. Results: Our hypotheses regarding effectiveness and applicability have been accepted in the context of the study. The initial design guidelines were refined throughout the study. Thus, we also contribute concrete guidelines for architecting distributed data-intensive systems with eventually consistent data. The guidelines are an advancement of Domain-Driven Design and provide additional patterns for the tactical design part. Conclusions: Based on our results, we recommend using the guidelines to architect safe eventually consistent systems. Because of the relevance of distributed data-intensive systems, we will drive this research forward and evaluate it in further domains.

Requirements for data storage and processing have reached new levels, with applications relying on the analysis of large amounts of data in order to support everyday life services to end users. Since the costs of maintaining and managing databases are significant, change data capture (CDC) techniques can be used to determine which parts of a data source have changed, and thus assist in the management of large volumes of data in data warehouses. In this paper we investigate a number of CDC techniques suitable for NoSQL databases. CDC techniques can be used to track modifications in a source database, which later can be made available to a target database. Our base system and testbed are based on Apache Cassandra, which is a NoSQL database that offers high performance and scalability. Cassandra is combined with a MapReduce framework, which is used to implement the logic of each CDC technique and is suitable for highly distributed and parallel computing. This paper also presents both a functional comparison of the different CDC techniques, as well as a performance evaluation in a real testbed.

This article introduces how temporal data can be maintained and processed by utilizing Column-oriented NoSQL databases (CoNoSQLDBs). Although each column in a CoNoSQLDB can store multiple data versions with their corresponded timestamps, its implicit temporal interval representation can cause wrong or misleading results during temporal query processing. In consequence, the original table representation supported by CoNoSQLDBs is not suitable for storing temporal data. To maintain the temporal data in the CoNoSQLDB tables, two alternative table representations can be adopted, namely, explicit history representation (EHR) and tuple time-stamping representation (TTR) in which each tuple (data version) has an explicit temporal interval. For processing TTR, the temporal relational algebra is extended to TTRO operator model with minor modifications. For processing EHR, a novel temporal operator model called CTO is proposed. Both TTRO and CTO contain eight temporal data operators, namely, Union, Difference, Intersection, Project, Filter, Cartesian product, Theta-Join and Group by with a set of aggregation functions, such as SUM, AVG, MAX and etc. Moreover, the authors implement each temporal operator by utilizing MapReduce framework to indicate which temporal operator model is more suitable for temporal data processing in the context of CoNoSQLDBs.

This paper describes Orchid, a system that converts declarative mapping specifications into data flow specifications (ETL jobs) and vice versa. Orchid provides an abstract operator model that serves as a common model for both transformation paradigms; both mappings and ETL jobs are transformed into instances of this common model. As an additional benefit, instances of this common model can be optimized and deployed into multiple target environments. Orchid is being deployed inFastTrack, a data transformation toolkit in IBM Information Server.

Extract, Transform, and Load (ETL) processes physically integrate data from multiple, heterogeneous sources in a central repository referred to as data warehouse. Physically integrated data gets stale when source data is changed, hence periodic refreshes are required. For efficiency reasons data warehouses are typically refreshed incrementally, i.e. changes are captured at the sources and propagated to the data warehouse on a regular basis. Dedicated ETL processes referred to as incremental load processes are employed to extract changes from the sources, propagate the changes, and refresh the data warehouse incrementally. Changes required in the data warehouse are inferred from changes captured at the sources during change propagation. The creation of incremental load processes is a complex task reserved to trained ETL programmers. In this paper we review existing Change Data Capture (CDC) techniques and discuss limitations of different approaches. We further review existing techniques for refreshing data warehouses. We then present an approach for generating incremental load processes from abstract schema mappings.

Abstract One necessary factor in the success of data grid technology to share structured and semi‐structured data across organizational boundaries is the availability of convenient, easy‐to‐use information integration services that provide an application‐specific, integrated view over the relevant data sources. To deal with the many challenges of information integration in such a dynamic environment, the integrated handling of metadata is an essential factor. In this paper, we present the PALADIN integration framework, a platform providing the common infrastructure on top of which the different integration services can be built. Copyright © 2008 John Wiley & Sons, Ltd.

Abstract The successful adaptation of information integration techniques to the requirements of data Grids is essential for the proliferation of Grid technology. In addition to the well‐known problems encountered when integrating heterogeneous sources, the dynamic Grid environment introduces new challenges. This paper discusses the problem of data source discovery, i.e. the selection of the most useful data sources for a given information demand out of a possibly very large set of candidates. We introduce the concept of data source utility and emphasize the pivotal role of semantic correspondences or schema matches for utility. Different variants of concrete utility measures used in an advanced Grid data source registry are presented. Copyright © 2007 John Wiley & Sons, Ltd.

Advanced data models and knowledge models together with their powerful query and manipulation languages have already proven to be essential for systems that support non-standard applications such as engineering and knowledge-based application systems. In order to raise their usability and acceptability, it is overly important to provide adequate implementation techniques that guarantee extensible and efficient processing for this advanced DBMS scenario. In this paper we present design alternatives and implementation techniques for such kinds of advanced DBMS, strongly focussing on query and knowledge processing in client/server architectures. To discuss our considerations and implementation technologies, we refer to the knowledge-processing framework of the KBMS KRISYS, although our ideas are generally applicable to (advanced) DBMS.

A great variety of techniques has been developed to optimize and enhance query processing for relational, client/server, distribute~ parallel, and heterogeneous database systems (DBS).Baaed on that work and experience, we investigate how far those techniques are applicable to query processing in Knowledge Base Management Systems (KBMS).Our reference system is the KRISYS KBMS that consists of a knowledge-processing system at the client (clientbaaed processing) and a data-processing system at the server (database backend).We describe a unifying thrnework for query ,processing incorporating both processing systems (as realized in KRISYS).This allows to distribute and balance the amount of work done in the client and in the server.Based on an evaluation of that framework, severaf approaches to further enhance knowledge processing are reported.