Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

HMM-CPM: a cloud instance resource prediction method tracing the workload trends via hidden Markov model

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Accurate prediction of cloud resource instances is becoming increasingly important for public cloud users and cloud service providers, because it touches on the reasonable reservation of cloud resources with minimize costs. However, current methods do not predict the instance types of cloud resources based on the application workloads from users, and less consider the characteristics of workload data changes in the real-time prediction. To solve these problems, this paper proposes an application workload-dependent cloud resource instance prediction model to predict appropriate cloud instance resource usage in a timely manner. Firstly, we adopt a trend degree (TD) to classify all requested workloads into three types of wave trend patterns. Next, a Hidden Markov model based cloud resource prediction method (HMM-CPM) tracing the requested workload trends is presented. Finally, the reasonable cloud instance types following the patterns of the requested workloads can be predicted. The simulation results show that the proposed method can predict cloud resource instance types in the scenario with certain workload fluctuation, and the prediction accuracy is higher than the existing related approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

Enquiries about data availability should be directed to the authors.

References

  1. Kashyap, S., Singh, A.: Prediction-based scheduling techniques for cloud data center’s workload: a systematic review. Clust. Comput. 26(5), 3209–3235 (2023)

    Article  Google Scholar 

  2. Wang, X., Cao, J., Xiang, Y.: Dynamic cloud service selection using an adaptive learning mechanism in multi-cloud computing. J. Syst. Softw. 100, 195–210 (2015)

    Article  Google Scholar 

  3. Reiss, C., Tumanov, A., Ganger, G.R., Katz, R.H., Kozuch, M.A.: Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In: Acm Symposium on Cloud Computing (2012)

  4. Chen, Z., Hu, J., Min, G., Zomaya, A.Y., El-Ghazawi, T.: Towards accurate prediction for high-dimensional and highly-variable cloud workloads with deep learning. IEEE Trans. Parallel Distrib. Syst. 31(4), 923–934 (2020)

    Article  Google Scholar 

  5. Wang, X., Cao, J., Yang, D., Qin, Z., Buyya, R.: Online cloud resource prediction via scalable window waveform sampling on classified workloads. Futur. Gener. Comput. Syst. 117, 338–358 (2021)

    Article  Google Scholar 

  6. Yadav, M.P., Rohit, Yadav, D.K.: Resource provisioning through machine learning in cloud services. Arabtan J. Sci. Eng. 47(2), 1483–1505 (2022)

    Article  Google Scholar 

  7. Wei, L., Foh, C.H., He, B., Cai, J.: Towards efficient resource allocation for heterogeneous workloads in iaas clouds. IEEE Trans. Cloud Comput. 6(1), 264–275 (2018)

    Article  Google Scholar 

  8. Wei, L., He, B., Foh, C.H.: Towards multi-resource physical machine provisioning for iaas clouds. In: Jamalipour, A., Deng, D. (eds.) 2014 IEEE International Conference on Communications (ICC), pp. 3469–3474 (2014)

  9. Zhu, Y., Zhang, W., Chen, Y., Gao, H.: A novel approach to workload prediction using attention-based lstm encoder-decoder network in cloud environment. Eurasip J. Wirel. Commun. Netw. 2019(1), 274 (2019)

  10. Sampaio, A.M., Barbosa, J.G., Prodan, R.: Piasa: a power and interference aware resource management strategy for heterogeneous workloads in cloud data centers. Simul. Model. Pract. Theory 57, 142–160 (2015)

    Article  Google Scholar 

  11. Wang, Y., Yu, L., Teng, F., Song, J., Yuan, Y.: Resource load forecasting model based on long-short-term sequence feature fusion. J. Comput. Appl. 42(5), 1508–1515 (2022)

    Google Scholar 

  12. Hu, Y., Deng, B., Peng, F., Wang, D.: Workload prediction for cloud computing elasticity mechanism. In: 2016 IEEE International Conference on Cloud Computing and Big Data Analysis (ICCCBDA 2016), pp. 244–249 (2016)

  13. Kumar, J., Singh, A.K.: Workload prediction in cloud using artificial neural network and adaptive differential evolution. Futur. Gener. Comput. Syst. 81, 41–52 (2018)

    Article  Google Scholar 

  14. Khan, A., Yan, X., Tao, S., Anerousis, N.: Workload characterization and prediction in the cloud: a multiple time series approach. In: DeTurck, F., Gaspary, L., Medhi, D. (eds.) 2012 IEEE Network Operations and Management Symposium (NOMS), pp. 1287–1294 (2012)

  15. Wang, H., Panneerselvam, J., Liu, L., Lu, Y., Zhai, X., Ali, H.: Cloud workload analytics for real-time prediction of user request patterns. In: 2018 IEEE 20th International Conference on High Performance Computing and Communications (HPCC), pp. 1677–1684 (2018)

  16. Duggan, M., Mason, K., Duggan, J., Howley, E., Barrett, E.: Predicting host cpu utilization in cloud computing using recurrent neural networks. In: 2017 IEEE 12th International Conference for Internet Technology and Secured Transactions (ICITST), pp. 67–72 (2017)

  17. Huang, Z., Peng, J., Lian, H., Guo, J., Qiu, W.: Deep recurrent model for server load and performance prediction in data center. Complexity (2017). https://doi.org/10.1155/2017/8584252

    Article  Google Scholar 

  18. Xia, B., Li, T., Zhou, Q., Li, Q., Zhang, H.: An effective classification-based framework for predicting cloud capacity demand in cloud services. IEEE Trans. Serv. Comput. 14(4), 944–956 (2021)

    Article  Google Scholar 

  19. Wang, X., Cao, J., Buyya, R.: Adaptive cloud bundle provisioning and multi-workflow scheduling via coalition reinforcement learning. IEEE Trans. Comput. 72(4), 1041–1054 (2023)

    Article  Google Scholar 

  20. Hwang, R.-H., Lee, C.-N., Chen, Y.-R., Zhang-Jian, D.-J.: Cost optimization of elasticity cloud resource subscription policy. IEEE Trans. Serv. Comput. 7(4), 561–574 (2014)

    Article  Google Scholar 

  21. Gupta, S., Dileep, A.D., Gonsalves, T.A.: A joint feature selection framework for multivariate resource usage prediction in cloud servers using stability and prediction performance. J. Supercomput. 74(11, SI), 6033–6068 (2018)

    Article  Google Scholar 

  22. Pinciroli, R., Ali, A., Yan, F., Smirni, E.: Cedule+: Resource management for burstable cloud instances using predictive analytics. IEEE Trans. Netw. Serv. Manage. 18(1), 945–957 (2021)

    Article  Google Scholar 

  23. Gupta, S., Dileep, A.D., Gonsalves, T.A.: Online sparse blstm models for resource usage prediction in cloud datacentres. IEEE Trans. Netw. Serv. Manage. 17(4), 2335–2349 (2020)

    Article  Google Scholar 

  24. Nawrocki, P., Osypanka, P., Posluszny, B.: Data-driven adaptive prediction of cloud resource usage. J. Grid Comput. 21(1), 6 (2023)

  25. Wang, S., Yao, Y., Xiao, Y., Chen, H.: Dynamic resource prediction in cloud computing for complex system simulatiuon: a probabilistic approach using stacking ensemble learning. In: 2020 International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI), pp. 198–201 (2020)

  26. Shen, H., Chen, L.: A resource-efficient predictive resource provisioning system in cloud systems. IEEE Trans. Parallel Distrib. Syst. 33(12), 3886–3900 (2022)

    Article  Google Scholar 

  27. Rossi, A., Visentin, A., Prestwich, S., Brown, K.N.: Bayesian uncertainty modelling for cloud workload prediction. In: 2022 IEEE 15th International Conference on Cloud Computing (CLOUD), pp. 19–29 (2022)

  28. Bao, L., Yang, J., Zhang, Z., Liu, W., Chen, J., Wu, C.: On accurate prediction of cloud workloads with adaptive pattern mining. J. Supercomput. 79(1), 160–187 (2023)

    Article  Google Scholar 

  29. Liu, S., Jiang, C.: A novel prediction approach based on three-way decision for cloud datacenters. Appl. Intell. 53(17), 20239–20255 (2023)

    Article  Google Scholar 

  30. Amazon: Amazon ec2 on-demand pricing. https://aws.amazon.com/ec2/pricing/on-demand (2023)

  31. Wilkes, J.: Yet more Google compute cluster trace data. Google research blog, Mountain View, CA, USA. https://ai.googleblog.com/2020/04/yet-more-google-compute-cluster-trace.html (2020)

  32. LinearModels: Scikit-learn. https://scikit-learn.org/stable/modules/linear_model.html (2020)

  33. SupportVectorRegression: Scikit-learn. https://scikit-learn.org/stable/modules/svm.html (2020)

  34. Baig, S.U.R., Iqbal, W., Berral, J.L., Erradi, A., Carrera, D.: Adaptive prediction models for data center resources utilization estimation. IEEE Trans. Netw. Serv. Manage. 16(4), 1681–1693 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by the Natural Science Foundation of Shanghai Science and Technology Innovation Action Plan of China under Grant 22ZR1425300, the National Natural Science Foundation of China under Grant 61963017, the Shanghai Educational Science Research Project of China under Grant C2022056, the Shanghai Science and Technology Program of China under Grant 23010501000, and the Humanities and Social Sciences of Ministry of Education Planning Fund of China under Grant 22YJAZHA145.

Funding

The authors have not disclosed any funding.

Author information

Author notes

  1. Xiaogang Wang, Rongting Li and Yangli Liu contributed equally to this work.

Authors and Affiliations

  1. School of Electronic and Information, Shanghai Dianji University, 300 Shuihua Road, Pudong New Area District, Shanghai, 201306, People’s Republic of China

    Zhihe Yang, Xiaogang Wang, Rongting Li & Yangli Liu

Authors
  1. Zhihe Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaogang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rongting Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yangli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors confirm their contribution to the paper as follows: Yang Z.: Conceptualization, methodology, formal analysis, writing - original draft. Wang X.: Supervision, method guidance, writing - review and editing. Li R.: Algorithmic programming, data curation, validation. Liu Y.: Project coordination.

Corresponding author

Correspondence to Xiaogang Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

The interpretation on the content extension

The preliminary work of this paper has been published at 2023 9th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS2023) under the title “Cloud Instance Resources Prediction Based on Hidden Markov Model”, and we have substantially modified and expanded the preliminary version of this paper about the model, algorithms, examples, experimental content and references for forming this journal standard version.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Z., Wang, X., Li, R. et al. HMM-CPM: a cloud instance resource prediction method tracing the workload trends via hidden Markov model. Cluster Comput 27, 11823–11838 (2024). https://doi.org/10.1007/s10586-024-04580-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-024-04580-7

Keywords

Search

Navigation