Abstract
Database performance optimization has become a hot issue in recent years. Some works deeply reconstruct the database to achieve specified goals like throughput or latency. The others focus on the database’s configuration knobs with reinforcement learning (RL) to improve the performance without any empirical knowledge. But the exhaustive offline training process costs plenty of time and resources due to the large inefficient configuration knobs combinations with trial-and-error methods. The most time-consuming part of the process is not the RL network training, but the database performance evaluation for acquiring the reward values of target performance like throughput or latency. So we propose an expert database tuning system (XTuning) which contains a correlation knowledge model to remove unnecessary training costs and a multi-instance mechanism (MIM) to support fine-grained tuning for diverse workloads. The models define the importance and correlations among these configuration knobs for the user’s specified target. Then we implement the models as Progressive Expert Knowledge Tuning (PEKT) algorithm with an abstracted architectural optimization integrated into XTuning. Experiments show that XTuning can effectively reduce the training time and achieves extra performance promotion compared with the state-of-the-art tuning methods.
This work is supported by the National Key Research and Development Program of China (No. 2019YFE0198600), National Natural Science Foundation of China (No. 61972402, 61972275, and 61732014).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Balmau, O., Dinu, F., Zwaenepoel, W., Gupta, K., Chandhiramoorthi, R., Didona, D.: SILK: preventing latency spikes in log-structured merge key-value stores. In: 2019 USENIX Annual Technical Conference, pp. 753–766, July 2019
Chai, Y., Chai, Y., Wang, X., Wei, H., Wang, Y.: Adaptive lower-level driven compaction to optimize LSM-tree key-value stores. IEEE Trans. Knowl. Data Eng. (2020, early access). https://doi.org/10.1109/TKDE.2020.3019264
Dageville, B., et al.: The snowflake elastic data warehouse. In: Proceedings of the 2016 International Conference on Management of Data, pp. 215–226 (2016)
Dai, Y., et al.: From wisckey to bourbon: a learned index for log-structured merge trees. In: 14th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 20), pp. 155–171 (2020)
Dayan, N., Athanassoulis, M., Idreos, S.: Monkey: optimal navigable key-value store. In: SIGMOD, pp. 79–94. ACM (2017)
Dayan, N., Idreos, S.: Dostoevsky: better space-time trade-offs for LSM-tree based key-value stores via adaptive removal of superfluous merging. In: Proceedings of the 2018 International Conference on Management of Data, pp. 505–520 (2018)
Dong, S., Callaghan, M., Galanis, L., Borthakur, D., Savor, T., Strum, M.: Optimizing space amplification in RocksDB. In: CIDR (2017)
Huang, D., et al.: TiDB: a raft-based HTAP database. Proc. VLDB Endowment 13(12), 3072–3084 (2020)
Hugegraph (2021). https://github.com/hugegraph/hugegraph
Li, G., Zhou, X., Li, S., Gao, B.: Qtune: a query-aware database tuning system with deep reinforcement learning. Proc. VLDB Endowment 12(12), 2118–2130 (2019)
Matsunobu, Y., Dong, S., Lee, H.: Myrocks: LSM-tree database storage engine serving facebook’s social graph. Proc. VLDB Endowment 13(12), 3217–3230 (2020)
Nebula graph (2021). https://github.com/vesoft-inc/nebula-graph
Taft, R., et al.: Cockroachdb: the resilient geo-distributed sql database. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pp. 1493–1509 (2020)
Van Aken, D., Pavlo, A., Gordon, G.J., Zhang, B.: Automatic database management system tuning through large-scale machine learning. In: Proceedings of the 2017 ACM International Conference on Management of Data, pp. 1009–1024 (2017)
YCSB-C (2018). https://github.com/basicthinker/YCSB-C
Zhang, J., et al.: An end-to-end automatic cloud database tuning system using deep reinforcement learning. In: Proceedings of the 2019 International Conference on Management of Data, pp. 415–432 (2019)
Zhu, Y., et al.: Bestconfig: tapping the performance potential of systems via automatic configuration tuning. In: Proceedings of the 2017 Symposium on Cloud Computing, pp. 338–350 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chai, Y., Ge, J., Chai, Y., Wang, X., Zhao, B. (2021). XTuning: Expert Database Tuning System Based on Reinforcement Learning. In: Zhang, W., Zou, L., Maamar, Z., Chen, L. (eds) Web Information Systems Engineering – WISE 2021. WISE 2021. Lecture Notes in Computer Science(), vol 13080. Springer, Cham. https://doi.org/10.1007/978-3-030-90888-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-90888-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90887-4
Online ISBN: 978-3-030-90888-1
eBook Packages: Computer ScienceComputer Science (R0)