In large federations of autonomous database systems, automatic distribution of the query workload to those systems is a critical issue. We examine this problem under the perspective of microeconomics theory and show how the latter can be used to construct an efficient decentralized mechanism that maximizes system throughput. In particular, we introduce a solution that is based on the notion of query markets. We examine the properties of these markets and show that they result in Pareto-optimal allocations of resources to queries. An extensive set of experiments with both a simulator and an actual implementation on top of a commercial DBMS demonstrate significant improvements in the overall system throughput when our technique is used.

Large-scale distributed environments, where each node is completely autonomous and offers services to its peers through external communication, pose significant challenges to query processing and optimization. Autonomy is the main source of the problem, as it results in lack of knowledge about any particular node with respect to the information it can produce and its characteristics, for example, cost of production or quality of produced results. In this article, inspired by e-commerce technology, we recognize queries as commodities and model query optimization as a trading negotiation process. Subquery answers and subquery operator execution jobs are traded between nodes until deals are struck with some nodes for all of them. Such trading may also occur recursively, in the sense that some nodes may play the role of intermediaries between other nodes (subcontracting). We identify the key parameters of the overall framework and suggest several potential alternatives for each one. In comparison to trading negotiations for e-commerce, query optimization faces unique new challenges that stem primarily from the fact that queries have a complex structure and can be broken into smaller parts. We address these challenges through a particular instantiation of our framework focusing primarily on the optimization algorithms run on “buying” and “selling” nodes, the evaluation metrics of the queries, and the negotiation strategy. Finally, we present the results of several experiments that demonstrate the performance characteristics of our approach compared to those of traditional query optimization.

An object oriented statistical metadata model is presented, which can be used in building information systems providing metadata-guided, statistical data processing features. The semantics of the model are analyzed and a set of operators (transformations) is proposed that allows for the automatic manipulation of both data and metadata at the same time. We discuss the mathematical properties of these transformations, and subsequently as a case study, we demonstrate how a statistical office can use the presented framework to build a Web site offering ad hoc query capabilities to its data consumers.