Many index advisors have recently been proposed to build indexes automatically to improve query performance. However, they mainly consider performance improvement in static scenarios. Their robustness, i.e., stable performance in dynamic scenarios (e.g., with minor workload changes), has not been well investigated. This paper addresses the challenges of assessing the index advisor's robustness from the following aspects. First, we introduce perturbation-based workloads for robustness assessment and identify three typical perturbation constraints that occur in real scenarios. Second, with the perturbation constraints, we formulate the generation of perturbed queries as a sequence-to-sequence problem and propose TRAP (Tailored Robustness assessment via Adversarial Perturbation) to pinpoint the performance loopholes of index advisors. Third, to generalize to various index advisors, we place TRAP in an opaque-box setting (i.e., with little knowledge of the index advisors' internal design), and we propose a two-phase training paradigm to efficiently train TRAP without elaborately annotated data. Fourth, we conduct comprehensive robustness assessments on standard benchmarks and real workloads for ten existing index advisors. Our findings reveal that these index advisors are vulnerable to the workloads generated by TRAP. Finally, based on the assessment results, we shed light on insights to enhance the robustness of different index advisors. For example, learning-based index advisors can benefit from adopting a fine-grained state representation and a candidate pruning strategy.

Despite the promising performance of recent learning-based Index Advisors (IAs), they exhibited the robustness issue when poisoning attacks polluted training data. This paper presents the first attempt to study the robustness of updatable learning-based IAs against poisoning attack, i.e., whether the IAs can maintain robust performance if their training/updating is disturbed by injecting an extraneous toxic workload. The goal is to provide an opaque-box stress test that is generally effective in evaluating the robustness of different learning-based IAs without using the users' private data. There are three challenges, i.e., how to probe "index preference" from opaque-box IAs, how to design effective injecting strategies even if the IAs can be fine-tuned, and how to generate queries to meet the specific constraints for IA probing and injecting. The presented stress-test framework PIPA consists of a probing stage, an injecting stage, and a query generator. To address the first challenge, the probing stage estimates the IA's indexing preference by observing its responses to the probing workload. To address the second challenge, the injecting stage injects workloads that spoof the IA to demote the top-ranked indexes in the estimated indexing preference and promote mid-ranked indexes. The stress test is effective because the IA is trapped in a local optimum even after fine-tuning. To address the third challenge, PIPA utilizes IABART (Index Aware BART) to generate queries that can be optimized by building indexes on a given set of indexes. Extensive experiments on different benchmarks against various learning-based IAs demonstrate the effectiveness of PIPA and that existing learning-based IAs are non-robust when faced with even a subtle amount of injected extraneous toxic workloads.

Current index advisors often struggle to balance efficiency and effectiveness when dealing with workload shifts. This arises from ignorance of the continual similarity and distant variety in workloads. This paper proposes a novel learning-based index advisor called BALANCE, which boosts indexing performance by leveraging knowledge obtained from dynamic and heterogeneous workloads. Our approach consists of three components. First, we build separate Lightweight Index Advisors (LIAs) on sequential chunks of similar workloads, where each LIA is trained with a small batch of workloads drawn from the chunk, and it provides direct index recommendations for all workloads in the same chunk. Second, we perform a policy transfer mechanism by adapting the LIA's index selection strategy from historical knowledge, substantially reducing the training overhead. Third, we employ a self-supervised contrastive learning method to provide an off-the-shelf workload representation, enabling the LIA to generate more accurate index recommendations. Extensive experiments across various benchmarks demonstrate that BALANCE improves the state-of-the-art learning-based index advisor, SWIRL, by 10.03% while reducing training overhead by 35.70% on average.

Index advisors have become an essential tool to optimize index selection and accelerate query processing. Various index advisors have been developed in recent years, and comprehensively assessing their performance from multiple aspects is necessary. In this demonstration, we introduce VITA, a user-friendly and informative tool for interactively Visualizing, Testing, and Analyzing index advisors. For a user-given workload, VITA can visualize the main steps of the index selection procedure in ten existing index advisors to facilitate the management of index advisors. Moreover, VITA can assess the index advisor's robustness w.r.t. workload drift by generating testing workloads, i.e., potentially future workloads that may damage the index advisor's performance. Finally, VITA provides a comparative analysis across index advisors on four aspects, including the index advisor's utility (i.e., the ratio of the reduced workload cost), robustness (i.e., the performance under dynamic workload), overhead (i.e., the time to acquire the final configuration), and scalability (i.e., the volume of the enumerated index candidates). Therefore, VITA can thoroughly compare existing index advisors to help users determine the most suitable index advisor that meets their requirements. VITA is now being integrated into the openGauss platform as a plug-in.

Indexes are vital to enhance the lookup on single or multiple columns, and building proper indexes can significantly improve the database performance. Existing works focus on adding new indexes that can benefit the read queries, but they have several limitations. First, real-world workloads may have numerous queries and it is tricky to analyze their index requirements and find the most beneficial indexes within resource limit. Second, they fail to consider the update of existing indexes, which may be redundant or even have negative effects to current workload. Third, they cannot estimate the index maintenance costs, which are affected by multiple index utilization factors and can significantly affect the index benefits, especially for high-write-ratio workloads. To address those challenges, we propose an incremental index management system Autoindex for dynamic workloads. First, to support incremental index management, we map the incoming queries into query templates and efficiently generate promising candidate indexes from matched templates. And then we propose to utilize Monte Carlo Tree Search to incrementally add indexes from the candidate indexes or remove indexes from existing indexes, so as to ensure high workload performance. Besides, we propose a deep index estimation model, which integrates the practical experience to extract critical cost features and applies deep regression to estimate index benefits from historical index management data. We have implemented the modules like candidate index generation and index estimator in an open-sourced database system openGauss. Experimental re-sults showed that our method outperformed existing approaches on both testing and real-world workloads.

Although learning-based database optimization techniques have been studied from academia in recent years, they have not been widely deployed in commercial database systems. In this work, we build an autonomous database framework and integrate our proposed learning-based database techniques into an open-source database system openGauss. We propose effective learning-based models to build learned optimizers (including learned query rewrite, learned cost/cardinality estimation, learned join order selection and physical operator selection) and learned database advisors (including self-monitoring, self-diagnosis, self-configuration, and self-optimization). We devise an effective validation model to validate the effectiveness of learned models. We build effective training data management and model management platforms to easily deploy learned models. We have evaluated our techniques on real-world datasets and the experimental results validated the effectiveness of our techniques. We also provide our learnings of deploying learning-based techniques.

We demonstrate a self-driving system DBMind, which provides three autonomous capabilities in database, including self-monitoring, self-diagnosis and self-optimization. First, self-monitoring judiciously collects database metrics and detects anomalies (e.g., slow queries and IO contention), which can profile database status while only slightly affecting system performance (<5%). Then, self-diagnosis utilizes an LSTM model to analyze the root causes of the anomalies and automatically detect root causes from a pre-defined failure hierarchy. Next, self-optimization automatically optimizes the database performance using learning-based techniques, including deep reinforcement learning based knob tuning, reinforcement learning based index selection, and encoder-decoder based view selection. We have implemented DBMind in an open source database openGauss and demonstrated real scenarios.