HPC Container Management at the University of Virginia
Article No.: 73, Pages 1 - 4
Abstract
We present the management of containers on our HPC cluster through EasyBuild and Apptainer. Containers are installed as modules by customizing the EasyBuild framework so that users can easily navigate the available containers as they would for non-container modules. We make use of the module load message to show how to run the container and a GPU property tag for GPU-enabled containers. In addition, we utilize containers for a subset of JupyterLab kernels through our Open OnDemand HPC web portal. Our container deployment workflow and user training are then discussed.
References
[1]
Markus Geimer, Kenneth Hoste, and Robert McLay. 2014. Modern Scientific Software Management Using EasyBuild and Lmod. In 2014 First International Workshop on HPC User Support Tools. 41–51. https://doi.org/10.1109/HUST.2014.8
[2]
Lisa Gerhardt, Wahid Bhimji, Shane Canon, Markus Fasel, Doug Jacobsen, Mustafa Mustafa, Jeff Porter, and Vakho Tsulaia. 2017. Shifter: Containers for HPC. Journal of Physics: Conference Series 898, 8 (oct 2017), 082021. https://doi.org/10.1088/1742-6596/898/8/082021
[3]
Matt et. al Heon, Dan Walsh, Brent Baude, Urvashi Mohnani, Ashley Cui, Tom Sweeney, Giuseppe Scrivano, Chris Evich, Valentin Rothberg, Miloslav Trmač, Jhon Honce, Qi Wang, Lokesh Mandvekar, Adrian Reber, Eduardo Santiago, Sascha Grunert, Nalin Dahyabhai, Anders Bjorklund, Kunal Kushwaha, Sujil Ashwin Sha, Yiqiao Pu, Zhangguanzhang, Matej Vasek, and Podman Community. 2021. Podman - : A tool for managing OCI containers and pods. https://doi.org/10.5281/zenodo.4735634
[4]
Gregory M. Kurtzer, Vanessa Sochat, and Michael W. Bauer. 2017. Singularity: Scientific containers for mobility of compute. PLOS ONE 12, 5 (05 2017), 1–20. https://doi.org/10.1371/journal.pone.0177459
[5]
Nina Mujkanovic, Juan J. Durillo, Nicolay Hammer, and Tiziano Müller. 2023. Survey of adaptive containerization architectures for HPC. In Proceedings of the SC ’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis(SC-W ’23). Association for Computing Machinery, New York, NY, USA, 165–176. https://doi.org/10.1145/3624062.3624588
[6]
Reid Priedhorsky and Tim Randles. 2017. Charliecloud: unprivileged containers for user-defined software stacks in HPC. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, Colorado) (SC ’17). Association for Computing Machinery, New York, NY, USA, Article 36, 10 pages. https://doi.org/10.1145/3126908.3126925
[7]
[7] Singularity Developers 2021. https://doi.org/10.5281/zenodo.1310023
[8]
Yucheng Zhang, Lev Gorenstein, Payas Bhutra, and Ryan T DeRue. 2022. Containerized Bioinformatics Ecosystem for HPC. In 2022 IEEE/ACM International Workshop on HPC User Support Tools (HUST). 1–10. https://doi.org/10.1109/HUST56722.2022.00006
Index Terms
- HPC Container Management at the University of Virginia
Recommendations
Survey of adaptive containerization architectures for HPC
SC-W '23: Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and AnalysisContainers offer an array of advantages that benefit research reproducibility and portability. As container tools mature, container security improves, and high-performance computing (HPC) and cloud system tools converge, supercomputing centers are ...
The HPC Container Experience on the Summit Supercomputer
PEARC '23: Practice and Experience in Advanced Research Computing 2023: Computing for the Common GoodContainers are seeing widespread use in the world of High Performance Computing, with many HPC Centers either providing their own containerization solution or adopting existing ones like Singularity and Apptainer. The demand for containerization options ...
Multiple HPC Environments-Aware Container Image Configuration Workflow for Large-Scale All-to-All Protein–Protein Docking Calculations
Supercomputing FrontiersAbstractContainers offer considerable portability advantages across different computing environments. These advantages can be realized by isolating processes from the host system whilst ensuring minimum performance overhead. Thus, use of containers is ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
608 pages
ISBN:9798400704192
DOI:10.1145/3626203
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 July 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
PEARC '24: Practice and Experience in Advanced Research Computing
July 21 - 25, 2024
RI, Providence, USA
Acceptance Rates
Overall Acceptance Rate 133 of 202 submissions, 66%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 147Total Downloads
- Downloads (Last 12 months)147
- Downloads (Last 6 weeks)50
Reflects downloads up to 24 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in