|
1.
|
EOS software evolution enabling LHC Run 3
/ Caffy, Cedric (CERN) ; Amadio, Guilherme (CERN) ; Guenther, Jaroslav (CERN) ; Lekshmanan, Abhishek (CERN) ; Mascetti, Luca (CERN) ; Peters, Andreas (CERN) ; Reis, Manuel (CERN) ; Simon, Michal Kamil (CERN) ; Sindrilaru, Elvin (CERN) ; Smith, David (CERN)
EOS has been the main storage system at CERN for more than a decade, continuously improving in order to meet the ever evolving requirements of the LHC experiments and the whole physics user community. In order to satisfy the demands of LHC Run-3, in terms of storage performance and tradeoff between cost and capacity, EOS was enhanced with a set of new functionalities and features that we will detail in this paper.First of all, we describe the use of erasure coded layouts in a large-scale deployment which enables an efficient use of available storage capacity, while at the same time providing end-users with better throughput when accessing their data. [...]
2024 - 8 p.
- Published in : EPJ Web Conf. 295 (2024) 01022
Fulltext: PDF;
In : 26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023, pp.01022
|
|
|
2.
|
Operation of the CERN disk storage infrastructure during LHC Run-3
/ Caffy, Cedric (CERN) ; Amadio, Guilherme (CERN) ; Rios, Maria Arsuaga (CERN) ; Reis, Manuel Batalha (CERN) ; Capitoni, Niccolo (CERN) ; Contescu, Cristian (CERN) ; Guenther, Jaroslav (CERN) ; Mascetti, Luca (CERN) ; Peters, Andreas (CERN) ; Sindrilaru, Elvin (CERN) et al.
The CERN IT Storage group operates multiple distributed storage systems to support all CERN data storage requirements. The storage and distribution of physics data generated by LHC and non-LHC experiments is one of the biggest challenges the group has to take on during LHC Run-3.EOS [1], the CERN distributed disk storage system is playing a key role in LHC data-taking. [...]
2024 - 8 p.
- Published in : EPJ Web Conf. 295 (2024) 01041
Fulltext: PDF;
In : 26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023, pp.01041
|
|
|
3.
|
FTS Service Evolution and LHC Run-3 Operations
/ Murray, Steven (CERN) ; Patrascoiu, Mihai (CERN) ; Mascetti, Luca (CERN) ; Lopes, Joao Pedro (CERN) ; Misra, Shubhangi (CERN) ; Junior, Eraldo Silva (Rio de Janeiro, CBPF)
The File Transfer Service (FTS) is a software system responsible for queuing, scheduling, dispatching and retrying file transfer requests, it is used by three of the LHC experiments, namely ATLAS, CMS and LHCb, as well as non LHC experiments including AMS, Dune and NA62. FTS is critical to the success of many experiments and the service must remain available and performant during the entire LHC Run-3. [...]
2024 - 8 p.
- Published in : EPJ Web Conf. 295 (2024) 01031
Fulltext: PDF;
In : 26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023, pp.01031
|
|
|
4.
|
|
|
5.
|
|
|
6.
|
EOS deployment on Ceph RBD/CephFS with K8s
/ Fornari, Federico (INFN, CNAF) ; Costantini, Alessandro (INFN, CNAF) ; Cavalli, Alessandro (INFN, CNAF) ; Cesini, Daniele (INFN, CNAF) ; Duma, Doina Cristina (INFN, CNAF) ; Falabella, Antonio (INFN, CNAF) ; Fattibene, Enrico (INFN, CNAF) ; Mascetti, Luca (CERN) ; Morganti, Lucia (INFN, CNAF) ; Peters, Andreas-Joachim (CERN) et al.
The present activity focused on the integration of different storage systems (EOS [1] and Ceph [2]) with the aim to combine the high level scalability and stability of EOS services with the reliability and redundancy features provided by Ceph. The work has been carried out as part of the collaboration between the national center of INFN (Italian Institute for Nuclear Physics) dedicated to Research and Development on Information and Communication Technologies and the Conseil Européen pour la Recherche Nucléaire (CERN), with the goal of evaluating and testing different technologies for next-generation storage challenges. [...]
2023 - 5 p.
- Published in : J. Phys. : Conf. Ser. 2438 (2023) 012032
Fulltext: PDF;
In : 20th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2021), Daejeon, Korea, 29 Nov - 3 Dec 2021, pp.012032
|
|
|
7.
|
|
EOS for Users - how to use the CERN physics storage system most effectively
/ Valverde Cameselle, Roberto (speaker) (CERN) ; Mascetti, Luca (speaker) (CERN) ; Sindrilaru, Elvin Alin (speaker) (CERN) ; Peters, Andreas Joachim (speaker) (CERN)
The IT storage group provides end-user access to a 700 PB disk storage system: CERN EOS.
In this seminar we will explain your possibilities to use EOS storage as a CERN user most effectively for everyone!
**Part 1 :** Dive into the EOS eco-system
We will start with a brief introduction:
*How is the EOS service deployed and segmented? How do you get access to EOS storage and how you can authenticate to the service?*
We will explain the various access interfaces:
- Command line access using the shell
- Using EOS as a filesystem /eos/
- Remote access protocols root:// and https://
- Accessing EOS from applications like ROOT, C++, Python …
- The CERNBox web interface
*You will learn, how you share access to files, folders or subtrees with your colleagues, how the permission systems of EOS and CERNBox interact, how you get an EOS drive on your Mac, Linux or Windows computer, how you can verify your quota, how you can understand where you use most of your space, how you can access EOS from outside CERN, what are the best access method for applications and many more useful hints for your daily work.*
We will finish with a short list of features, which are configured to mitigate user errors and service downtimes:
-Service & Data high-availability model
-Backup system
-File Versioning
-Undo Deletion using the EOS recycle bin
*If you deleted all your files, how can you get them back? Can you?*
**Part 2:** Running workflows using EOS storage
This part will cover best practices for running interactive and batch workflows using EOS storage with few examples on a laptop/desktop, lxplus, the batch farm etc.
*What can you do to get efficient data access and what you should never do! How can you authenticate from GITLAB to EOS?*
We will also briefly give some insights, how EOS service managers might influence or change your access to EOS..
2023 - 5048.
HEP Computing; EOS 2023 Workshop
External links: Talk details; Event details
In : EOS 2023 Workshop
|
|
|
8.
|
|
|
9.
|
LHC Data Storage: Preparing for the Challenges of Run-3
/ Arsuaga-Rios, Maria (CERN) ; Bahyl, Vladimír (CERN) ; Batalha, Manuel (CERN) ; Caffy, Cédric (CERN) ; Cano, Eric (CERN) ; Capitoni, Niccolo (CERN) ; Contescu, Cristian (CERN) ; Davis, Michael (CERN) ; Alvarez, David Fernandez (CERN) ; Guenther, Jaroslav (CERN) et al.
The CERN IT Storage Group ensures the symbiotic development and operations of storage and data transfer services for all CERN physics data, in particular the data generated by the four LHC experiments (ALICE, ATLAS, CMS and LHCb).In order to accomplish the objectives of the next run of the LHC (Run-3), the Storage Group has undertaken a thorough analysis of the experiments’ requirements, matching them to the appropriate storage and data transfer solutions, and undergoing a rigorous programme of testing to identify and solve any issues before the start of Run-3.In this paper, we present the main challenges presented by each of the four LHC experiments. We describe their workflows, in particular how they communicate with and use the key components provided by the Storage Group: the EOS disk storage system; its archival back-end, the CERN Tape Archive (CTA); and the File Transfer Service (FTS). [...]
2021 - 10 p.
- Published in : EPJ Web Conf. 251 (2021) 02023
Fulltext: PDF;
In : 25th International Conference on Computing in High-Energy and Nuclear Physics (CHEP), Online, Online, 17 - 21 May 2021, pp.02023
|
|
|
10.
|
CERN Disk Storage Services: Report from last data taking, evolution and future outlook towards Exabyte-scale storage
/ Mascetti, Luca (CERN) ; Rios, Maria Arsuaga (CERN) ; Bocchi, Enrico (CERN) ; Vicente, Joao Calado (CERN) ; Cheong, Belinda Chan Kwok (CERN) ; Castro, Diogo (CERN) ; Collet, Julien (CERN) ; Contescu, Cristian (CERN) ; Labrador, Hugo Gonzalez (CERN) ; Iven, Jan (CERN) et al.
The CERN IT Storage group operates multiple distributed storage systems to support all CERN data storage requirements: the physics data generated by LHC and non-LHC experiments; object and file storage for infrastructure services; block storage for the CERN cloud system; filesystems for general use and specialized HPC clusters; content distribution filesystem for software distribution and condition databases; and sync&share; cloud storage for end-user files. The total integrated capacity of these systems exceeds 0.6 Exabyte.Large-scale experiment data taking has been supported by EOS and CASTOR for the last 10+ years. [...]
2020 - 7 p.
- Published in : EPJ Web Conf. 245 (2020) 04038
Fulltext from publisher: PDF;
In : 24th International Conference on Computing in High Energy and Nuclear Physics, Adelaide, Australia, 4 - 8 Nov 2019, pp.04038
|
|