Parallel processing is the key to speedup performance and to achieve high throughput in processing large scale data analytical workloads. However, failures of nodes involved in the analytical query can interrupt the whole process, resulting in the complete restart of the query if the system does not

Hive is a data warehousing solution on top of the Hadoop MapReduce framework that has been designed to handle large amounts of data and store them in tables like a relational database management system or a conventional data warehouse while using the parallelization and batch processing functionalities of the Hadoop MapReduce framework to speed up the execution of queries. Data inserted into Hive is stored in the Hadoop FileSystem (HDFS), which is part of the Hadoop MapReduce framework. To make the data accessible to the user, Hive uses a query language similar to SQL, which is called HiveQL. When a query is issued in HiveQL, it is translated by a parser into a query execution plan that is optimized and then turned into a series of map and reduce iterations. These iterations are then executed on the data stored in the HDFS, writing the output to a file.

Presents the welcome message from the conference proceedings.

Stream data analysis differs significantly from traditional data processing. To process the data online the algorithm has to work in one pass, incorporating new data into a model maintained in main memory. Storing a model or synopsis of processed data in the memory, which we call "data compression", is an important technique in both incremental and differential stream mining. This paper proposes several data compression schemes in one-pass categorical data clustering, and demonstrates their performance on synthetic and real data. Our compression schemes can efficiently generate compact representations of original data, so as to enable the algorithm to process streams at high speed and detect the changes in underlying data. The example algorithm based on these compression schemes achieves good accuracy in short execution time.

The visual exploration of large databases calls for a tight coupling of database and visualization systems. Current visualization systems typically fetch all the data and organize it in a scene tree, which is then used to render the visible data. For immersive data explorations, where an observer navigates in a potentially huge data space and explores selected data regions this approach is inadequate. A scalable approach is to make the database system observer-aware and exchange the data that is visible and most relevant to the observer.In this paper we present iTopN an incremental algorithm for extracting the most visible objects relative to the current position of the observer. We implement iTopN and compare it to an improved version of the R-tree that extends LRU with the caching of the top levels of the R-tree (LW-LRU). Our experiments show that iTopN is orders of magnitude faster than LW-LRU given the same amount of memory. Our experiments also show that for LW-LRU to perform as fast as iTopN it needs three times as much memory.

As applications enabled by the Internet become information rich, ensuring access to quality information in the presence of potentially malicious entities will be a major challenge. Denial of information (DoI) attacks attempt to degrade the quality of information by deliberately introducing noise that appears to be useful information. The mere availability of information is insufficient if the user must find a needle in a haystack of noise that is created by an adversary to hide critical information. We focus on the characterization of information quality metrics that are relevant in the presence of DoI attacks. In particular, two complementary metrics are explored. Information regularity captures predictability in the patterns of information creation and access. The second metric, information quality trust, captures the known ability of an information source to meet the needs of its clients.

A system that enables a person to program without typing is needed because of the high incidents of repetitive stress injuries among people who program. This paper presents a design for a system that generates environments that enables people to program by voice and a method of determining if the system is successful. It also shows how this generator can be used to support entering data and writing XML documents.

Transaction-time databases support access to not only the current database state, but also previous database states. Supporting access to previous database states requires large quantities of data and necessitates efficient temporal query processing techniques. Previously, we presented a log based storage structure and algorithms for the differential computation of previous database states. Timeslices-i.e., previous database states-are computed by traversing a log of database changes, using previously computed and cached timeslices as outsets. When computing a new timeslice, the cache will contain two candidate outsets: an earlier outset and a later outset. The new timeslice can be computed by either incrementally updating the earlier outset or decrementally "downdating" the later outset using the log. The cost of this computation is determined by the size of the log between the outset and the new timeslice. The paper proposes an efficient algorithm that identifies the cheaper outset for the differential computation. The basic idea is to compute the sizes of the two pieces of the log by maintaining and using a tree structure on the timestamps of the database changes in the log. The lack of a homogeneous node structure, a controllable and high fill factor for nodes, and of appropriate node allocation in existing tree structures (e.g., B/sup +/ trees, Monotonic B/sup +/ trees, and Append only trees) render existing tree structures unsuited for our use. Consequently, a specialized tree structure, the pointer-less insertion tree, is developed to support the algorithm. As a proof of concept, we have implemented a main memory version of the algorithm and its tree structure.

Views can be computed by means of recomputation or they can be computed incrementally. Incremental view computation outperforms view recomputation in many situations. This is particularly true in distributed database environments such as data warehousing environments. The authors present and analyze two algorithms for the incremental computation of relational set difference views. Set differences occur naturally in the definition of the set-division operator and in rewritten nested queries. The I/O efficiency of the incremental algorithms is compared to each other and to efficient recomputation algorithms. The cost analysis strongly indicates that incremental computation of set difference views outperforms recomputation of set difference views in many situations.

We present and analyze algorithms for the incremental computation of time-varying queries in which selection predicates refer to the state of a clock. Such queries occur naturally in many situations where temporal data are processed. Incremental techniques for query computation have proven to be more efficient than other techniques in many situations. However, all existing incremental techniques for query computation assume that old query results remain valid if no intermediate changes are made to the underlying database. Unfortunately, this assumption does not hold for time-varying queries whose results may change just because time passes. In order to solve this problem, we introduce the notion of a superview which contains all current tuples that will eventually satisfy the selection predicate of a time-varying selection. Based on the notion of superview, we develop efficient algorithms for the incremental computation of time-varying selections. Our algorithms, combined with existing incremental algorithms, allow complex time-varying queries to benefit from the proven efficiency of incremental techniques. It is important to notice that without our algorithms, the existing algorithms for incremental computation would be useless for any time-varying query expression.>

Efficient algorithms for incrementally computing nested query expressions do not exist. Nested query expressions are query expressions in which selection/join predicates contain subqueries. In order to respond to this problem, we propose a two-step strategy for incrementaly computing nested query expressions. In step (1), the query expression is transformed into an equivalent unnested flat query expression. In step (2), the flat query expression is incrementally computed. To support step (1), we have developed a very concise algebra-to-algebra transformation algorithm, and we have formally proved its correctness. The flat query expressions resulting from the transformation make intensive use of the relational set-difference operator. To support step (2), we present and analyze an efficient algorithm for incrementally computing set differences based on view pointer caches. When combined with existing incremental algorithms for SPJ queries, our incremental set-difference algorithm can be used to compute the unnested flat query expressions efficiently. It is important to notice that without our incremental set-difference algorithm the existing incremental algorithms for SPJ queries are useless for any query involving the set-difference operator, including queries that are not the result of unnesting nested queries.

A graphical representation tool-updated Petri nets (UPN)-has been developed to model rule-base specifications for CIM databases. UPN facilitates the modeling of relationships between operations of various manufacturing application systems and the database updates and retrievals among all the respective distributed databases. Based on this representation, a hierarchical modeling technique which includes refining and aggregating rules has also been developed. An application of the UPN is demonstrated in designing rule-based systems for controlling and integrating the information flow between manufacturing applications, including computer aided design, computer aided process planning, manufacturing resources planning, and shop floor control.>

Focuses on the development of a methodology within a software environment for automating the rule-based implementation of specifications of integrated manufacturing information systems. The specifications are initially formulated in a natural language and subsequently represented in terms of a graphical representation by the system designer. A new graphical representation tool is based on updated Petri nets (UPN) that we have developed as a specialized version of colored Petri nets. The rule-based implementation approach utilizes the similarity of features between UPN and the general rule specification language used for the implementation. The automation of the translation of UPN to the rule specification language is expected to considerably reduce the life-cycle for design and implementation of the system. The application presented deals with the control and management of information flow between the computer-aided design, process planning, manufacturing resource planning and shop floor control databases. This provides an integrated information framework for computer integrated manufacturing systems.>

Abstract An incrementally maintained mapping from a network to a relational database is presented. This mapping may be established either to support the efficient retrieval of data from a network database through a relational interface, or as the first step in a gradual conversion of data and applications from a network to a relational database system. After the mapping has been established, the only data mapped from the network to the relational database are the increments resulting from updates on the network database. The mapping is therefore an efficient alternative to mappings that repeatedly map the results of retrievals through the relational interface from the network database to the relational database. This is in particular the case when the two databases reside on different hosts. Applications on the network database may, under certain restrictions, gradually be moved to the relational database, while the mapping incrementally maintains the relational database for the applications that remain on the network database. A detailed, but generic, account of how to build such a mapping from a network to a relational database is given, including all the algorithms needed and examples of their use.

A data model that allows for the storage of detailed change history in so-called backlog relations is described. Its extended relational algebra, in conjunction with the extended data structures, provides a powerful tool for the retrieval of patterns and exceptions in change history. An operator, Sigma , based on the notion of compact active domain is introduced. It groups data not in predefined groups but in groups that fit the data. This operator further expands the retrieval capabilities of the algebra. The expressive power of the algebra is demonstrated by examples, some of which show how patterns and exceptions in change history can be detected. Sample applications of this work are statistical and scientific databases, monitoring (of databases, manufacturing plants, power plants, etc.), CAD, and CASE.>

Programming languages and databases have evolved on separate foundations and with separate goals for many years. Buzzwords such as persistent data objects, objectoriented databases and software engineering databases reflect increased activity aimed at integrating the two areas at their current state of evolution. This paper suggests that we retum to basics and consider the foundation of the two areas, grammars and relations, as a basis for their integration. We present an algorithm, GeneRel, which given a grammar automatically generates a set of normalized relations in which objects derived from the grammar can be stored. We demonstrate how the algorithm applied to a meta- grammar generates relations in which grammars derived from the meta-grammar can be stored. We show how an extended relational algebra can be used to retrieve information about stored grammars and stored objects derived from the grammars. We outline an algorithm, GeneView, which given a set of non-terminals from a grammar generates a set of view definitions on the created relations.

An implementation model for the standard relational data model extended with transaction time is presented. The implementation model integrates techniques of view materialization, differential computation, and deferred update into a coherent whole. It is capable of storing any view (reflecting past or present states) and subsequently using stored views as outsets for incremental and decremental computations of requested views, making it more flexible than previously proposed partitioned storage models. The working and the expressiveness of the model are demonstrated by sample queries that show how historical data are retrieved.>

Commercially available database systems do not meet the information and processing needs of design and manufacturing environments. A new generation of systems-engineering information systems-must be built to meet these needs. The architectural and computational aspects of such systems are addressed, and solutions are proposed. The authors argue that a mainframe-workstation architecture is needed to provide distributed functionality while ensuring high availability and low communication overhead, that explicit control of metaknowledge is needed to support extendibility and evolution, that large rule bases are needed to make the knowledge of the systems active, and that incremental computation models are needed to achieve the required performance of such engineering information systems.>