Abstract
In high performance computing (HPC) job schedulers usually divide resources of computing nodes into slots. Each slot can be assigned to execute only a single job from the queue. In some cases, jobs do not fully utilize all available resources from the slot which leads to internal fragmentation, wasted resources and to an increase of queue wait time. In this paper, we propose fair resource allocation strategies that can be applied in job schedulers for resource allocation. We cover such resources as CPU time, residential memory and network bandwidth.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kuchumov, R., Petrunin, V., Korkhov, V., Balashov, N., Kutovskiy, N., Sokolov, I.: Design and implementation of a service for cloud HPC computations. In: Gervasi, O., et al. (eds.) ICCSA 2018. LNCS, vol. 10963, pp. 103–112. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95171-3_9
Kuchumov, R.I., Korkhov, V.V.: Design and implementation of a service for performing HPC computations in cloud environment. In: CEUR Workshop Proceedings, vol. 2267, pp. 233–236 (2018)
Korkhov, V., Kobyshev, S., Degtyarev, A., Bogdanov, A.: Light-weight cloud-based virtual computing infrastructure for distributed applications and hadoop clusters. In: Gervasi, O., et al. (eds.) ICCSA 2017. LNCS, vol. 10408, pp. 399–411. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-62404-4_29
Wong, C.S., Tan, I.K.T., Kumari, R.D., Lam, J.W., Fun, W.: Fairness and interactive performance of o(1) and CFS Linux kernel schedulers. In: International Symposium on Information Technology, vol. 4, pp. 1–8. IEEE, August 2008
Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple Linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3
SGE, Son of Grid Engine. https://arc.liv.ac.uk/trac/SGE. Accessed 07 May 2019
Maui: 5.2 Node Allocation, Page Redirection. http://docs.adaptivecomputing.com/maui/5.2nodeallocation.php. Accessed 07 May 2019
LSF: Fairshare Scheduling. https://www.bsc.es/support/LSF/9.1.2/lsf_admin/index.htm?chap_fairshare_lsf_admin.html~main. Accessed 07 May 2019
Sedighi, A., Deng, Y., Zhang, P.: Fariness of task scheduling in high performance computing environments. Scalable Comput. Pract. Experience 15(3), 271–288 (2014)
Wong, C.S., Tan, I., Kumari, R.D., Wey, F.: Towards achieving fairness in the Linux scheduler. ACM SIGOPS Operat. Syst. Rev. 42(5), 34–43 (2008)
Waldspurger, C.A.: Memory resource management in VMware ESX server. ACM SIGOPS Oper. Syst. Rev. 36(SI), 181–194 (2002)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Kuchumov, R., Korkhov, V. (2019). Fair Resource Allocation for Running HPC Workloads Simultaneously. In: Misra, S., et al. Computational Science and Its Applications – ICCSA 2019. ICCSA 2019. Lecture Notes in Computer Science(), vol 11622. Springer, Cham. https://doi.org/10.1007/978-3-030-24305-0_55
Download citation
DOI: https://doi.org/10.1007/978-3-030-24305-0_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-24304-3
Online ISBN: 978-3-030-24305-0
eBook Packages: Computer ScienceComputer Science (R0)