Anomaly detection system plays an important role in maintaining the stability of cloud database. Existing studies mainly focus on significant deviations in multivariate time series, such as a combination of CPU utilization, transactions per second, etc, to detect abnormal issues. Due to the complexity of cloud database structure and functions, these approaches are difficult to achieve a balance among detection performance, detection efficiency and workload adaptability. In this paper, we propose DBCatcher, a cloud database online anomaly detection system based on indicator correlation. Through extensive analysis of real-world cloud database time series, we find the correlations among trends in the same key performance indicators across databases within the same unit, which inspires us to explore a time series correlation measurement method that can efficiently detect abnormal issues. Meanwhile, we design a flexible time window observation mechanism and an adaptive threshold learning policy to minimize misjudgment caused by key performance indicator fluctuations, greatly enhancing the detection performance and workload adaptability. We conduct extensive experiments under real-world and synthetic workloads. Experimental results show that DBCatcher significantly improves the detection performance and detection efficiency compared to existing methods.

Recently, using machine learning for performance tuning of cloud database (CDB) service has shown great potentials. However, facing personalized requirements such as various restrictions for tuning with very different workloads, pre-trained models may mismatch or recommend suboptimal configurations given a new workload. On the other hand, if the system tunes configurations in an online fashion, the system will suffer from the cold start problem, resulting in long tuning time and performance fluctuation. To accommodate these problems, we propose an online CDB tuning system called HUNTER. The key feature of HUNTER is a hybrid architecture, which uses samples generated by Genetic Algorithm to warm-start the finer grained exploration of deep reinforcement learning. Meanwhile, we employ Principal Component Analysis, Random Forest, and Fast Exploration Strategy to reduce the search space and the update time of the learning model. In addition, we further propose a clone and parallelization scheme to stress-test workloads on multiple cloned CDB instances (CDBs), resulting in faster and safer configuration exploration. Extensive trials on CDB with public and real-world workloads demonstrate that, given the same time budget and resources, HUNTER improves performance and considerably decreases recommendation time compared to state-of-the-art tuning systems, with accelerations of up to 2.8× and 22.8× utilizing 1 and 20 cloned CDBs, respectively.

Abstract Configuration tuning is vital to optimize the performance of a database management system (DBMS). It becomes more tedious and urgent for cloud databases (CDB) due to diverse database instances and query workloads, which make the job of a database administrator (DBA) very difficult. Existing solutions for automatic DBMS configuration tuning have several limitations. Firstly, they adopt a pipelined learning model but cannot optimize the overall performance in an end-to-end manner. Secondly, they rely on large-scale high-quality training samples which are hard to obtain. Thirdly, existing approaches cannot recommend reasonable configurations for a large number of knobs to tune whose potential values live in such high-dimensional continuous space. Lastly, in cloud environments, existing approaches can hardly cope with the changes of hardware configurations and workloads, and have poor adaptability. To address these challenges, we design an end-to-end automatic CDB tuning system, $${\texttt {CDBTune}}^{+}$$ CDBTune + , using deep reinforcement learning (RL). $${\texttt {CDBTune}}^{+}$$ CDBTune + utilizes the deep deterministic policy gradient method to find the optimal configurations in a high-dimensional continuous space. $${\texttt {CDBTune}}^{+}$$ CDBTune + adopts a trial-and-error strategy to learn knob settings with a limited number of samples to accomplish the initial training, which alleviates the necessity of collecting a massive amount of high-quality samples. $${\texttt {CDBTune}}^{+}$$ CDBTune + adopts the reward-feedback mechanism in RL instead of traditional regression, which enables end-to-end learning and accelerates the convergence speed of our model and improves the efficiency of online tuning. Besides, we propose effective techniques to improve the training and tuning efficiency of $${\texttt {CDBTune}}^{+}$$ CDBTune + for practical usage in a cloud environment. We conducted extensive experiments under 7 different workloads on real cloud databases to evaluate $${\texttt {CDBTune}}^{+}$$ CDBTune + . Experimental results showed that $${\texttt {CDBTune}}^{+}$$ CDBTune + adapts well to a new hardware environment or workload, and significantly outperformed the state-of-the-art tuning tools and DBA experts.

Configuration tuning is vital to optimize the performance of database management system (DBMS). It becomes more tedious and urgent for cloud databases (CDB) due to the diverse database instances and query workloads, which make the database administrator (DBA) incompetent. Although there are some studies on automatic DBMS configuration tuning, they have several limitations. Firstly, they adopt a pipelined learning model but cannot optimize the overall performance in an end-to-end manner. Secondly, they rely on large-scale high-quality training samples which are hard to obtain. Thirdly, there are a large number of knobs that are in continuous space and have unseen dependencies, and they cannot recommend reasonable configurations in such high-dimensional continuous space. Lastly, in cloud environment, they can hardly cope with the changes of hardware configurations and workloads, and have poor adaptability. To address these challenges, we design an end-to-end automatic CDB tuning system, CDBTune, using deep reinforcement learning (RL). CDBTune utilizes the deep deterministic policy gradient method to find the optimal configurations in high-dimensional continuous space. CDBTune adopts a try-and-error strategy to learn knob settings with a limited number of samples to accomplish the initial training, which alleviates the difficulty of collecting massive high-quality samples. CDBTune adopts the reward-feedback mechanism in RL instead of traditional regression, which enables end-to-end learning and accelerates the convergence speed of our model and improves efficiency of online tuning. We conducted extensive experiments under 6 different workloads on real cloud databases to demonstrate the superiority of CDBTune. Experimental results showed that CDBTune had a good adaptability and significantly outperformed the state-of-the-art tuning tools and DBA experts.