Abstract
Serverless computing has emerged as a compelling paradigm for deploying applications and services due to its elastic scalability in response to changing demand. However, it often suffers from cold-start problems due to the overhead of initializing code and data dependencies. Thus, it’s necessary to keep a container warm after the container completes function processing and reuse it for subsequent requests. However, existing schedulers of Serverless platforms, which are usually load balancers, may not efficiently locate the reusable containers. Moreover, elastic scalers only scale passively based on predefined thresholds, making it challenging to handle burst requests. This paper introduces a scheduling algorithm called Consistent Hash-based Affinity Scheduling (CHAS), which aims to increase the chances of reusing warm containers by assigning functions to appropriate working nodes. We also propose a container prewarming strategy called LSTM-NB that uses a Long Short-Term Memory Network (LSTM) to predict the parameters of the negative binomial distribution (NB). This strategy performs joint learning of call time series of multiple functions and predicts future function calls to actively warm up or evict containers, thereby reducing cold start latency and excessive resource consumption. We build a serverless computing environment by using the SimPy discrete-event simulation framework to evaluate the proposed method. The results show that CHAS can reduce the cold start rate by an average of 10.5% compared to baseline approaches. Furthermore, an average reduction of 20.1% in the cold start rate can be achieved by prewarming with the LSTM-NB strategy.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aws lambda. https://aws.amazon.com/lambda/, (Accessed 14 Apr 2023)
Azure functions. https://azure.microsoft.com/en-us/products/functions/, (Accessed 14 Apr 2023)
Kubernetes. https://kubernetes.io/docs/concepts/services-networking/service/, (Accessed 15 Apr 2023)
Openwhisk. https://azure.microsoft.com/en-us/products/functions/, (Accessed 14 Apr 2023)
Simpy. https://pythonhosted.org/SimPy/, (Accessed 14 Apr 2023)
Abad, C.L., Boza, E.F., Van Eyk, E.: Package-aware scheduling of faas functions. In: Companion of the 2018 ACM/SPEC International Conference on Performance Engineering, pp. 101–106 (2018)
Aumala, G., Boza, E., Ortiz-Avilés, L., Totoy, G., Abad, C.: Beyond load balancing: package-aware scheduling for serverless platforms. In: 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), pp. 282–291. IEEE (2019)
Bauer, A., Grohmann, J., Herbst, N., Kounev, S.: On the value of service demand estimation for auto-scaling. In: German, R., Hielscher, K.-S., Krieger, U.R. (eds.) MMB 2018. LNCS, vol. 10740, pp. 142–156. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-74947-1_10
Dang-Quang, N.M., Yoo, M.: Deep learning-based autoscaling using bidirectional long short-term memory for kubernetes. Appl. Sci. 11(9), 3835 (2021)
Fan, D., He, D.: Knative autoscaler optimize based on double exponential smoothing. In: 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), pp. 614–617. IEEE (2020)
Fuerst, A., Sharma, P.: Faascache: keeping serverless computing alive with greedy-dual caching. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 386–400 (2021)
Herbst, N., et al.: Ready for rain? a view from spec research on the future of cloud metrics. arXiv preprint arXiv:1604.03470 (2016)
Imdoukh, M., Ahmad, I., Alfailakawi, M.G.: Machine learning-based auto-scaling for containerized applications. Neural Comput. Appl. 32, 9745–9760 (2020)
Karger, D., et al.: Web caching with consistent hashing. Comput. Netw. 31(11–16), 1203–1213 (1999)
Li, F., Hu, B.: Deepjs: job scheduling based on deep reinforcement learning in cloud data center. In: Proceedings of the 4th International Conference on Big Data and Computing, pp. 48–53 (2019)
Mampage, A., Karunasekera, S., Buyya, R.: Deadline-aware dynamic resource management in serverless computing environments. In: 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pp. 483–492. IEEE (2021)
Manner, J., Endreß, M., Heckel, T., Wirtz, G.: Cold start influencing factors in function as a service. In: 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), pp. 181–188. IEEE (2018)
Mitzenmacher, M.: The power of two choices in randomized load balancing. IEEE Trans. Parallel Distrib. Syst. 12(10), 1094–1104 (2001)
Shahrad, M., et al.: Serverless in the wild: characterizing and optimizing the serverless workload at a large cloud provider. In: 2020 USENIX annual technical conference (USENIX ATC 20), pp. 205–218 (2020)
Suo, K., Son, J., Cheng, D., Chen, W., Baidya, S.: Tackling cold start of serverless applications by efficient and adaptive container runtime reusing. In: 2021 IEEE International Conference on Cluster Computing (CLUSTER), pp. 433–443. IEEE (2021)
Suresh, A., Somashekar, G., Varadarajan, A., Kakarla, V.R., Upadhyay, H., Gandhi, A.: Ensure: efficient scheduling and autonomous resource management in serverless environments. In: 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS), pp. 1–10. IEEE (2020)
Vahidinia, P., Farahani, B., Aliee, F.S.: Mitigating cold start problem in serverless computing: a reinforcement learning approach. IEEE Internet Things J. 10(5), 3917–3927 (2022)
Wu, S., et al.: Container lifecycle-aware scheduling for serverless computing. Software: Pract. Experience 52(2), 337–352 (2022)
Xu, Z., Zhang, H., Geng, X., Wu, Q., Ma, H.: Adaptive function launching acceleration in serverless computing platforms. In: 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS), pp. 9–16. IEEE (2019)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 62072216 and No. 62372214) and the Science and Technology Development Fund, Macau SAR (File no. 0076/2022/A2).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chen, S., Li, G., Dai, C., Li, W., Zhao, Q. (2024). Joint Optimization of Request Scheduling and Container Prewarming in Serverless Computing. In: Tari, Z., Li, K., Wu, H. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2023. Lecture Notes in Computer Science, vol 14487. Springer, Singapore. https://doi.org/10.1007/978-981-97-0834-5_10
Download citation
DOI: https://doi.org/10.1007/978-981-97-0834-5_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0833-8
Online ISBN: 978-981-97-0834-5
eBook Packages: Computer ScienceComputer Science (R0)