Abstract
Microservices deployment in the cloud often faces a prevalent challenge: how to maximize resource utilization while maintaining high quality-of-service (QoS). Existing automatic scaling tools frequently exhibit limited adaptability, particularly when handling frequent request load fluctuations, which exacerbates the challenge. To address this issue, we introduce a proactive runtime deployment optimization method for multi-stage microservices, aiming to ensure both resource efficiency and QoS.
Our proposed method encompasses four interrelated modules–forecasting, constraint planning, judgment selection, and execution–which collaboratively work towards optimizing runtime resource allocation, generating viable deployment plans, and identifying cost-efficient solutions without compromising QoS. Through a set of experiments, we demonstrate that the proposed proactive deployment optimization method can potentially reduce computational resource usage by 35% while maintaining the desired quality of service.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alenizi, A., Ammar, R., Elfouly, R., Alsulami, M.: Queue analysis for probabilistic cloud workflows. In: 2020 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), pp. 1–6 (2020). https://doi.org/10.1109/ISSPIT51521.2020.9408967
Alsarhan, A., Itradat, A., Al-Dubai, A.Y., Zomaya, A.Y., Min, G.: Adaptive resource allocation and provisioning in multi-service cloud environments. IEEE Trans. Parallel Distrib. Syst. 29(1), 31–42 (2018)
Baccelli, F., Bremaud, P.: Elements of Queueing Theory. Elements of Queueing Theory (1961)
Bhardwaj, S., Sahoo, B.: A particle swarm optimization approach for cost effective saas placement on cloud. In: International Conference on Computing, Communication and Automation, ICCCA 2015, pp. 686–690 (2015). https://doi.org/10.1109/CCAA.2015.7148462
Chainbi, W., Sassi, E.: A multiswarm for composite saas placement optimization based on pso. Softw.- Pract. Exp. 48(10), 1847–1864 (2018). https://doi.org/10.1002/spe.2600
Chatfield, C.: The holt-winters forecasting procedure. J. Roy. Stat. Soc. 27(3), 264–279 (1978). https://doi.org/10.2307/2347162
Dikaiakos, M.D., Katsaros, D., Mehra, P., Pallis, G., Vakali, A.: Cloud computing: distributed internet computing for it and scientific research. IEEE Internet Comput. 13(5), 10–11 (2009). https://doi.org/10.1109/MIC.2009.103
Fu, K., Zhang, W., Chen, Q., Zeng, D., Guo, M.: Adaptive resource efficient microservice deployment in cloud-edge continuum. IEEE Trans. Parallel Distrib. Syst. 33(8), 1825–1840 (2022). https://doi.org/10.1109/TPDS.2021.3128037
Gan, Y., Liang, M., Dev, S., Lo, D., Delimitrou, C.: Sage: practical and scalable ml-driven performance debugging in microservices. In: International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS, pp. 135–151 (2021). https://doi.org/10.1145/3445814.3446700
Gan, Y., et al.: An open-source benchmark suite for cloud and iot microservices. arXiv: 1905.11055 (2019)
Gan, Y., et al.: Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices. In: International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS, pp. 19–33 (2019). https://doi.org/10.1145/3297858.3304004
Gardner, E.S.: Exponential smoothing: The State of the Art-part ii. Int. J. Forecast. 22(4), 637–666 (2006)
Gevros, P., Crowcroft, J.: Distributed resource management with heterogeneous linear controls. Comput. Netw. 45(6), 835–858 (2004)
Gias, A.U., Casale, G., Woodside, M.: Atom: model-driven autoscaling for microservices. In: Proceedings International Conference on Distributed Computing Systems 2019, pp. 1994–2004 (2019). https://doi.org/10.1109/ICDCS.2019.00197
Hajji, M.A., Mezni, H.: A composite particle swarm optimization approach for the composite saas placement in cloud environment. Soft. Comput. 22(12), 4025–4045 (2018). https://doi.org/10.1007/s00500-017-2613-8
He, X., Tu, Z., Wagner, M., Xu, X., Wang, Z.: Online deployment algorithms for microservice systems with complex dependencies. IEEE Trans. Cloud Comput. (2022). https://doi.org/10.1109/TCC.2022.3161684
Hyndman, R.J., Koehler, A.B., Snyder, R.D., Grose, S.: A state space framework for automatic forecasting using exponential smoothing methods. Int. J. Forecast. 18(3), 439–454 (2002)
Jia, R., Yang, Y., Grundy, J., Keung, J., Li, H.: A deadline constrained preemptive scheduler using queuing systems for multi-tenancy clouds. In: IEEE International Conference on Cloud Computing, CLOUD 2019, pp. 63–67 (2019)
Kannan, R.S., Subramanian, L., Raju, A., Ahn, J., Mars, J., Tang, L.: Grandslam: guaranteeing slas for jobs in microservices execution frameworks. In: Proceedings of the 14th EuroSys Conference 2019 pp. ACM Special Interest Group on Operating Systems (SIGOPS) (2019). https://doi.org/10.1145/3302424.3303958
Khazaei, H., Mii, J., Mii, V.B.: Modelling of cloud computing centers using m/g/m queues. In: Proceedings - International Conference on Distributed Computing Systems, pp. 87–92 (2011). https://doi.org/10.1109/ICDCSW.2011.13
Khazaei, H., Misic, J., Misic, V.B.: Performance analysis of cloud computing centers using m/g/m/m+r queuing systems. IEEE Trans. Parallel Distrib. Syst. 23(5), 936–943 (2012)
Klevans, R.L., Stewart, W.J.: From queueing networks to markov chains: the xmarca interface. Perform. Eval. 24(1–2), 23–45 (1995)
Liao, W.H., Chen, P.W., Kuai, S.C.: A resource provision strategy for software-as-a-service in cloud computing. Proc. Comput. Sci. 110, 94–101 (2017). https://doi.org/10.1016/j.procs.2017.06.123
Merkel, D.: Docker: lightweight linux containers for consistent development and deployment. Linux J. 2014(239) (2014). https://doi.org/10.5555/2600239.2600241
Moens, H., Truyen, E., Walraven, S., Joosen, W., Dhoedt, B., De Turck, F.: Cost-effective feature placement of customizable multi-tenant applications in the cloud. J. Netw. Syst. Manage. 22(4), 517–558 (2014)
Mohammadi, M., Jolai, F., Rostami, H.: An m/m/c queue model for hub covering location problem. Math. Comput. Model. 54(11–12), 2623–2638 (2011). https://doi.org/10.1016/j.mcm.2011.06.038
Pallis, G.: Cloud computing: the new frontier of internet computing. IEEE Internet Comput. 14(5), 70–73 (2010). https://doi.org/10.1109/MIC.2010.113
Qiu, H., Banerjee, S.S., Jha, S., Kalbarczyk, Z.T., Iyer, R.K.: Firm: an intelligent fine-grained resource management framework for slo-oriented microservices. Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2020, pp. 805–825 (2020). https://doi.org/10.48550/arXiv.2008.08509
Reiss, C., Tumanov, A., Ganger, G.R., Katz, R.H., Kozuch, M.A.: Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In: Proceedings of the 3rd ACM Symposium on Cloud Computing, SoCC 2012. ACM Special Interest Group on Management of Data (SIGMOD) (2012)
Tournaire, T., Castel-Taleb, H., Hyon, E., Hoche, T.: Generating optimal thresholds in a hysteresis queue: application to a cloud model. In: Proceedings - IEEE Computer Society’s Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS 2019, 283–294 (2019). https://doi.org/10.1109/MASCOTS.2019.00040
Villamizar, M., et al.: Infrastructure cost comparison of running web applications in the cloud using aws lambda and monolithic and microservice architectures. In: Proceedings - 2016 16th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2016, pp. 179–182 (2016). https://doi.org/10.1109/CCGrid.2016.37
Wada, H., Suzuki, J., Oba, K.: Queuing theoretic and evolutionary deployment optimization with probabilistic slas for service oriented clouds. In: SERVICES 2009–5th 2009 World Congress on Services (PART 1), pp. 661–669 (2009). https://doi.org/10.1109/SERVICES-I.2009.59
Wang, S., Li, X., Ruiz, R.: Performance analysis for heterogeneous cloud servers using queueing theory. IEEE Trans. Comput. 69(4), 563–576 (2020). https://doi.org/10.1109/TC.2019.2956505
Wu, H., Sun, Y., Wolter, K.: Analysis of the energy-response time tradeoff for delayed mobile cloud offloading. Perform. Eval. Rev. 43(2), 33–35 (2015). https://doi.org/10.1145/2825236.2825251
Yang, H., Chen, Q., Riaz, M., Luan, Z., Tang, L., Mars, J.: Powerchief: Intelligent power allocation for multi-stage applications to improve responsiveness on power constrained cmp. In: Proceedings - International Symposium on Computer Architecture Part F128643, pp. 133–146 (2017). https://doi.org/10.1145/3079856.3080224
Zhang, Y., Hua, W., Zhou, Z., Suh, G.E., Delimitrou, C.: Sinan: Ml-based and qos-aware resource management for cloud microservices. In: International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS, pp. 167–181 (2021). https://doi.org/10.1145/3445814.3446693
Zhou, H., et al.: Overload control for scaling wechat microservices. In: SoCC 2018 - Proceedings of the 2018 ACM Symposium on Cloud Computing, pp. 149–161 (2018). https://doi.org/10.1145/3267809.3267823
Zhou, X., et al.: Fault analysis and debugging of microservice systems: Industrial survey, benchmark system, and empirical study. IEEE Trans. Software Eng. 47(2), 243–260 (2021). https://doi.org/10.1109/TSE.2018.2887384
Acknowledgements
This research was funded by the Basic Public Welfare Research Project of Zhejiang Province grant number LY20F020014 and the National Science Foundation for Young Scientists of China grant number 61802096.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liao, P. et al. (2024). Efficient Proactive Resource Allocation for Multi-stage Cloud-Native Microservices. In: Tari, Z., Li, K., Wu, H. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2023. Lecture Notes in Computer Science, vol 14488. Springer, Singapore. https://doi.org/10.1007/978-981-97-0801-7_24
Download citation
DOI: https://doi.org/10.1007/978-981-97-0801-7_24
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0800-0
Online ISBN: 978-981-97-0801-7
eBook Packages: Computer ScienceComputer Science (R0)