We consider the problem of defining, generating, and tracing provenance in data-oriented workflows, in which input data sets are processed by a graph of transformations to produce output results. We first give a new general definition of provenance for general transformations, introducing the notions of correctness, precision, and minimality. We then determine when properties such as correctness and minimality carry over from the individual transformations' provenance to the workflow provenance. We describe a simple logical-provenance specification language consisting of attribute mappings and filters. We provide an algorithm for provenance tracing in workflows where logical provenance for each transformation is specified using our language. We consider logical provenance in the relational setting, observing that for a class of Select-Project-Join (SPJ) transformations, logical provenance specifications encode minimal provenance. We have built a prototype system supporting the features and algorithms presented in the paper, and we report a few preliminary experimental results.

Panda (for Provenance and Data), a system for data-oriented workflows that supports debugging and drill-down using logical provenance-provenance information stored at the processing-node level is demonstrated. In this demonstration, Panda is used to integrate, process, and analyze actual education data from multiple sources.

We consider a general workflow setting in which input data sets are processed by a graph of transformations to produce output results. Our goal is to perform efficient selective refresh of elements in the output data, i.e., compute the latest values of specific output elements when the input data may have changed. We explore how data provenance can be used to enable efficient refresh. Our approach is based on capturing one-level data provenance at each transformation when the workflow is run initially. Then at refresh time provenance is used to determine (transitively) which input elements are responsible for given output elements, and the workflow is rerun only on that portion of the data needed for refresh. Our contributions are to formalize the problem setting and the problem itself, to specify properties of transformations and provenance that are required for efficient refresh, and to provide algorithms that apply to a wide class of transformations and workflows. We have built a prototype system supporting the features and algorithms presented in the paper. We report preliminary experimental results on the overhead of provenance capture, and on the crossover point between selective refresh and full workflow recomputation.

We describe how aggregation is handled in the Trio system for uncertain and probabilistic data. Because “exact” aggregation in uncertain databases can produce exponentially sized results, we provide three alternatives: a low bound on the aggregate value, a high bound on the value, and the expected value. These variants return a single result instead of a set of possible results, and they are generally efficient to compute for both full-table and grouped aggregation queries. We provide formal definitions and semantics and a description of our open source implementation for single-table aggregation queries. We study the performance and scalability of our algorithms through experiments over a large synthetic data set. We also provide some preliminary results on aggregations over joins.

Most recommendation methods are "hard-wired" into the system and support only fixed recommendations. The purpose of this demo is to show the expressivity of flexible recommendation workflows, how flexible recommendations can be processed over relational data, and to show flexible recommendations in action through a real system used for course planning.

Objects are often organized in a hierarchy to help in managing or browsing them. For example, products in a store can be divided by type (electronics, clothes, books, ...) and then by brand (Sony, Epson, Dockers, ...). Web pages at a site can also be placed in a hierarchy. For instance, a French tourist site may have categories cities, history, hotels, tours; within the cities category we have pages divided by city, and then by events, maps, restaurants. In this paper we study the problem of hierarchy integration, in particular, how to combine two related hierarchies into one, more comprehensive one. The need to integrate arises in many situations where the objects come from different systems. In our product hierarchy examplFrench tourism sitese above, we may want to provide a comparison shopping service that offers products from two stores; in our tourism example, we may want to build a meta- web-site that combines the resources of two or more French tourism sites. To simplify the problem, we study how to merge one hierarchy into a second known base hierarchy, by copying references to objects into the base hierarchy, and perhaps by adding some categories into the base hierarchy.

CourseRank is a course planning tool aimed at helping students at Stanford. Recommendations comprise an integral part of it. However, implementing existing recommendation methods leads to fixed recommendations that cannot adapt to each particular student's changing requirements and do not help exploit the full extent of the available learning opportunities at the university. In this paper, we describe the concept of a flexible recommendation workflow, i.e., a high-level description of a parameterized process for computing recommendations. The input parameters of a flexible recommendation process comprise the "knobs" that control the final output and hence generate flexible recommendations. We describe how flexible recommendations can be expressed over a relational database and we present our prototype system that allows defining and executing different, fully-parameterized, recommendation workflows over relational data. Finally, we describe a user interface in CourseRank that allows students customize recommendations.