CERN Accelerating science

Published Articles
Title	Building and using containers at HPC centres for the ATLAS experiment
Author(s)	Benjamin, Douglas (Duke U.) ; Childers, Taylor (Argonne (main)) ; Lesny, David (Illinois U., Urbana) ; Oleynik, Danila (Texas U., Arlington) ; Panitkin, Sergey (Brookhaven) ; Tsulaia, Vakho (LBL, Berkeley) ; Yang, Wei (SLAC) ; Zhao, Xin (Brookhaven)
Collaboration	ATLAS Collaboration
Publication	EDP Sciences, 2019
Number of pages	8
In:	EPJ Web Conf. 214 (2019) 07005
In:	23rd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2018, Sofia, Bulgaria, 9 - 13 Jul 2018, pp.07005
DOI	10.1051/epjconf/201921407005
Subject category	Computing and Computers ; Particle Physics - Experiment
Accelerator/Facility, Experiment	CERN LHC ; ATLAS
Abstract	The HPC environment presents several challenges to the ATLAS experiment in running their automated computational workflows smoothly and efficiently, in particular regarding issues such as software distribution and I/O load. A vital component of the LHC Computing Grid, CVMFS, is not always available in HPC environments. ATLAS computing has experimented with all-inclusive containers, and later developed an environment to produce such containers for both Shifter and Singularity. The all-inclusive containers include most of the recent ATLAS software releases, database releases, and other tools extracted from CVMFS. This helped ATLAS to distribute software automatically to HPC centres with an environment identical to those in CVMFS. It also significantly reduced the metadata I/O load to HPC shared file systems. The production operation at NERSC has proved that by using this type of containers, we can transparently fit into the previously developed ATLAS operation methods, and at the same time scale up to run many more jobs.
Copyright/License	publication: © 2019-2024 the Authors (License: CC-BY-4.0)

Corresponding record in: Inspire