Abstract
Prediction of host load is essential in Cloud computing for improving resource utilization and achieving service-level agreements. However, accurate prediction of host load remains a challenge in Clouds because the type of load varies differently. Furthermore, selecting metrics for host load prediction is also a difficult task. With so many metrics in the Cloud systems, it is hard to determine which metrics are going to be useful. To address these challenges, this paper proposes an efficient deep learning model named Sibyl to improve the accuracy and efficiency of prediction. Sibyl includes two parts: a metrics selection module and a neural network training module. Sibyl first selects metrics by filtering out irrelevant metrics. Afterwards, Sibyl applies a powerful neural network model built with bidirectional long short-term memory to predict actual load one-step-ahead. We use Sibyl to analyze a 40-day load trace from a data center with 176 machines. Experiments show that Sibyl can reduce training metrics while maintaining prediction accuracy. Besides, Sibyl significantly improves prediction accuracy compared to other state-of-the-art methods based on autoregressive integrated moving-average and long short-term memory.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Nazarko, J., Jurczuk, A., Zalewski, W.: ARIMA models in load modelling with clustering approach. In: Proceedings of IEEE Russia Power Tech (2005)
Tran, N., Reed, D.A.: Automatic ARIMA time series modeling for adaptive I/O prefetching. In: Proceedings of International Conference on Supercomputing (2002)
Zhu, B., Sastry, S.: Revisit dynamic ARIMA based anomaly detection. In: Proceedings of Published in International Conference on Social Computing (2002)
Wang, J., Chen, J.W., Wang, Y., Zheng, D.: Intelligent load balancing strategies for complex distributed simulation applications. In: Proceedings of International Conference on Computational Intelligence and Security (2009)
Di, S., Kondo, D., Cirne, W.: Host load prediction in a google compute cloud with a Bayesian model. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (2012)
Dabrowski, C., Hunt, F.: Using Markov chain analysis to study dynamic behaviour in large-scale grid systems. In: Proceedings of the Seventh Australasian Symposium on Grid Computing (2009)
Akioka, S., Muraoka, Y.: Extended forecast of CPU and network load on computational grid. In: Proceedings of International Symposium on CLUSTER Computing and the Grid (2004)
Byun, E.J., Choi, S.J., Baik, M.S.: MJSA: markov job scheduler based on availability in desktop grid computing environment. Future Gener. Comput. Syst. 23(4), 616–622 (2007)
Yong, W.F, Goh, C., Hong, C.L., Zhan, Z.H., Li, Y.: Evolutionary neural network based energy consumption forecast for cloud computing. In: Proceedings of International Conference on Cloud Computing Research and Innovation (2015)
Xue, J., Yan, F., Birke, R., Chen, L.Y., Scherer, T.: PRACTISE: robust prediction of data center time series. In: Proceedings of International Conference on Network and Service Management (2015)
Lang, K., Zhang, M., Yuan, Y., Yue, X.: Short-term load forecasting based on multivariate time series prediction and weighted neural network with random weights and kernels. In: Proceedings of Cluster Computing (2018)
Huang, Z., Peng, J., Lian, H., Guo, J., Wei, Q.: Deep recurrent model for server load and performance prediction in data center. Complexity 2017(99), 1–10 (2007)
Saripalli, P., Oldenburg, C., Walters, B., Nanduri, R.: Implementation and usability evaluation of a cloud platform for scientific computing as a service (SCaaS). In: Proceedings of International Conference on Utility and Cloud Computing (2011)
Hussain, W., Hussain, F.K., Hussain, O., Chang, E.: Profile-based viable service level agreement (SLA) violation prediction model in the cloud. In: Proceedings of International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (2016)
Thalheim, J., Rodrigues, A., Akkus, I.E.: Sieve: actionable insights from monitored metrics in distributed systems. In: Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference (2017)
Amazon CloudWatch. https://aws.amazon.com/de/cloudwatch. Accessed 30 May 2018
Paparrizos, J., Gravano, L.: k-Shape: efficient and accurate clustering of time series. In: ACM SIGMOD International Conference on Management of Data, vol. 45, no. 1, pp. 1855–1870 (2015)
Ballesteros, M., Dyer, C., Smith, N.A.: Improved transition-based parsing by modeling characters instead of words with LSTMs. In: Proceedings of Empirical Methods in Natural Language Processing (2015)
Zhang, Y., Chen, G., Yu, D., Yaco, K.: Highway long short-term memory RNNS for distant speech recognition. In: Proceedings of International Conference on Acoustics, Speech and Signal Processing (2016)
Microsoft Cloud Monitoring. https://www.microsoft.com/en-us/cloud-platform/operations-management-suite. Accessed 28 May 2018
Google Stackdriver.https://cloud.google.com/stackdriver. Accessed 1 June 2018
Cao, J., Fu, J., Li, M.: CPU load prediction for cloud environment based on a dynamic ensemble model. Softw. Pract. Exp. 44(7), 793–804 (2014)
Song, B., Yu, Y., Zhou, Y.: Host load prediction with long short-term memory in cloud computing. J. Supercomput. 23(2), 1–15 (2017)
Acknowledgements
This work was supported by Grant 2017YFB 1010000 from the National Key R&D Program of China.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Z., Tang, X., Han, J., Wang, P. (2018). Sibyl: Host Load Prediction with an Efficient Deep Learning Model in Cloud Computing. In: Vaidya, J., Li, J. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2018. Lecture Notes in Computer Science(), vol 11335. Springer, Cham. https://doi.org/10.1007/978-3-030-05054-2_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-05054-2_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05053-5
Online ISBN: 978-3-030-05054-2
eBook Packages: Computer ScienceComputer Science (R0)