We demonstrate a generic and extensible cost-based optimization and execution system for XPath queries, named GeCOEX, using a comprehensive suite of query analyzing and administrative tools, named QuOAX. GeCOEX supports many different physical operator implementations and XML storage engines and is agnostic to the underlying physical data model. Its optimizer is the first generic cost-based optimizer for XPath queries that always picks the cheapest estimated plan, among a very large number of possible plans, for a wide range of XPath queries and different datasets in a very small fraction of the time required for efficient execution. The QuOAX suite provides administration tools that allow the user to add new -- or deactivate already deployed -- physical operator implementations, physical operator cost models and rewriting rules and also to make use of different XML storage and XML statistics estimators. QuOAX also provides query plan analysis and visualization tools that allow users to visualize the physical plan chosen by the optimizer or all possible generated physical plans for a given query and to execute any of those plans. QuOAX helps users to i) easily test new XPath processing techniques, comparing them directly with existing ones and identifying the situations to which they show promise, ii) improve the effectiveness of the optimizer and iii) find out the appropriate access methods or indices that are beneficial for a specific workload.

The creation of a generic and modular query optimization and processing infrastructure can provide significant benefits to XML data management. Key pieces of such an infrastructure are the physical operators that are available to the execution engine, to turn queries into execution plans. Such operators, to be efficient, need to implement sophisticated algorithms for logical XPath or XQuery operations. Moreover, to enable a cost-based optimizer to choose among them correctly, it is also necessary to provide cost models for such operator implementations. In this paper we present two novel families of algorithms for XPath physical operators, called LookUp (LU) and Sort-Merge-based (SM), along with detailed cost models. Our algorithms have significantly better performance compared to existing techniques over any one of a variety of different XML storage systems that provide a set of common primitive access methods. To substantiate the robustness and efficiency of our physical operators, we evaluate their individual performance over four different XML storage engines against operators that implement existing XPath processing techniques. We also demonstrate the performance gains for twig processing of using plans consisting of our operators compared to a state of the art holistic technique, specifically Twig2Stack. Additionally, we evaluate the precision of our cost models, and we conduct an analysis of the sensitivity of our algorithms and cost models to a variety of parameters.

We present a complete XPath cost-based optimization and execution framework and demonstrate its effectiveness and efficiency for a variety of queries and datasets. The framework is based on a logical XPath algebra with novel features and operators and a comprehensive set of rewriting rules that together enable us to algebraically capture many existing and novel processing strategies for XPath queries. An important part of the framework is PSA, a very efficient cost-based plan selection algorithm for XPath queries. In the presented experimental evaluation, PSA picked the cheapest estimated query plan in 100% of the cases. Our cost-based query optimizer independent of the underlying physical data model and storage system and of the available logical operator implementations, depending on a set of well-defined APIs. We also present an implementation of those APIs, including primitive access methods, a large pool of physical operators, statistics estimators and cost models, and experimentally demonstrate the effectiveness of our end-to-end query optimization system.

A lot of research has been conducted by the database community on methods and techniques for efficient XPath processing, with great success. Despite the progress made, significant opportunities for optimization of XPath still exist. One key to further improvements is to utilize more effectively existing facilities of relational RDBSes for the processing of XPath queries. After taking a comprehensive look at such facilities, we present techniques for XPath processing that work by identifying the best relational join algorithms, indices and file organization strategies for XPath queries. Our techniques both reduce the latency of the resulting SQL translations and guarantee their pipelined execution. We also propose a new technique for XML reconstruction from relations-mapped XML that "splits the difference" between schema-aware and schema-oblivious XML-to-relational mapping for a significant performance improvement. An extensive experimental study confirms the performance benefits of our optimization techniques and shows that a system implementing these techniques on top of a commercial RDBMS is competitive with respect to query performance with other native and relational-based state-of-the-art XPath processing systems, commercial as well as research prototypes.