Abstract
The trend towards transitioning from monolithic applications to microservices has been widely embraced in modern distributed systems and applications. This shift has resulted in the creation of lightweight, fine-grained, and self-contained microservices. Multiple microservices can be linked together via calls and inter-dependencies to form complex functions. One of the challenges in managing microservices is provisioning the optimal amount of resources for microservices in the chain to ensure application performance while improving resource usage efficiency. This paper presents ChainsFormer, a framework that analyzes microservice inter-dependencies to identify critical chains and nodes, and provision resources based on reinforcement learning. To analyze chains, ChainsFormer utilizes light-weight machine learning techniques to address the dynamic nature of microservice chains and workloads. For resource provisioning, a reinforcement learning approach is used that combines vertical and horizontal scaling to determine the amount of allocated resources and the number of replicates. We evaluate the effectiveness of ChainsFormer using realistic applications and traces on a real testbed based on Kubernetes. Our experimental results demonstrate that ChainsFormer can reduce response time by up to 26% and improve processed requests per second by 8% compared with state-of-the-art techniques.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Sock Shop: A Microservices Demo Application. https://microservices-demo.github.io/.
- 2.
Alibaba Cluster Trace Program: https://github.com/alibaba/clusterdata/tree/v2018.
References
Burns, B., Beda, J., Hightower, K.: Kubernetes: up and running: dive into the future of infrastructure. O’Reilly Media (2019)
Gan, Y., Liang, M., Dev, S., et al.: Sage: practical and scalable ml-driven performance debugging in microservices. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2021, pp. 135–151 (2021)
Gan, Y., Zhang, Y., Hu, K., et al.: Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices. In: Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systemsm ASPLOS 2019, pp. 19–33 (2019)
Hossen, M.R., Islam, M.A., Ahmed, K.: Practical efficient microservice autoscaling with qos assurance. In: Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2022, pp. 240–52 (2022)
Ioannidou, K., Nikolopoulos, S.D.: The longest path problem is polynomial on cocomparability graphs. Algorithmica 65, 177–205 (2013)
Kardani-Moghaddam, S., Buyya, R., Ramamohanarao, K.: Adrl: a hybrid anomaly-aware deep reinforcement learning-based resource scaling in clouds. IEEE Trans. Parallel Distrib. Syst. 32(3), 514–526 (2021)
Luo, S., Xu, H., Lu, C., et al.: An in-depth study of microservice call graph and runtime performance. IEEE Trans. Parallel Distrib. Syst. 33(12), 3901–3914 (2022)
Mirhosseini, A., Elnikety, S., Wenisch, T.F.: Parslo: a gradient descent-based approach for near-optimal partial slo allotment in microservices. In: Proceedings of the ACM Symposium on Cloud Computing, SoCC 2021, pp. 442–457 (2021)
Qiu, H., Banerjee, S.S., Jha, S., et al.: \(\{\)FIRM\(\}\): an intelligent fine-grained resource management framework for \(\{\)SLO-Oriented\(\}\) microservices. In: 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pp. 805–825 (2020)
Rzadca, K., Findeisen, P., Swiderski, J., et al.: Autopilot: workload autoscaling at google. In: Proceedings of the Fifteenth European Conference on Computer Systems. EuroSys 2020 (2020)
Wang, S., Guo, Y., Zhang, N., et al.: Delay-aware microservice coordination in mobile edge computing: a reinforcement learning approach. IEEE Trans. Mob. Comput. 20(3), 939–951 (2021)
Xu, M., Song, C., Ilager, S., et al.: Coscal: multifaceted scaling of microservices with reinforcement learning. IEEE Trans. Netw. Serv. Manage. 19(4), 3995–4009 (2022)
Xu, M., Song, C., Wu, H., et al.: Esdnn: deep neural network based multivariate workload prediction in cloud computing environments. ACM Trans. Internet Technol. 22(3) (2022)
Zhang, Y., Hua, W., Zhou, Z., et al.: Sinan: Ml-based and qos-aware resource management for cloud microservices. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2021, pp. 167–181 (2021)
Zhong, Z., Xu, M., Rodriguez, M., et al.: Machine learning-based orchestration of containers: A taxonomy and future directions. ACM Comput. Surv. 54(10s) (2022)
Acknowledgments
This work is supported by National Key R & D Program of China (No.2021YFB3300200), the National Natural Science Foundation of China (No. 62072451, 62102408), Shenzhen Industrial Application Projects of undertaking the National key R & D Program of China (No. CJGJZD20210408091600002), Shenzhen Science and Technology Program (No. RCBS20210609104609044), and Alibaba Group through Alibaba Innovative Research Program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Song, C. et al. (2023). ChainsFormer: A Chain Latency-Aware Resource Provisioning Approach for Microservices Cluster. In: Monti, F., Rinderle-Ma, S., Ruiz Cortés, A., Zheng, Z., Mecella, M. (eds) Service-Oriented Computing. ICSOC 2023. Lecture Notes in Computer Science, vol 14419. Springer, Cham. https://doi.org/10.1007/978-3-031-48421-6_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-48421-6_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48420-9
Online ISBN: 978-3-031-48421-6
eBook Packages: Computer ScienceComputer Science (R0)