|
1.
|
An HTTP REST API for Tape-backed Storage
/ Afonso, João (CERN) ; Caffy, Cédric (CERN) ; Patrascoiu, Mihai (CERN) ; Leduc, Julien (CERN) ; Davis, Michael (CERN) ; Murray, Steven (CERN) ; Cortes, Pablo (CERN)
The goal of the HTTP REST API for Tape project is to provide a simple, minimalistic and uniform interface to manage data transfers between Storage Endpoints (SEs) where the source file is on tape. The project is a collaboration between the developers of WLCG storage systems (EOS+CTA, dCache, StoRM) and data transfer clients (gfal2, FTS).For some years, HTTP has been growing in popularity as the preferred data transfer protocol between many WLCG SEs. [...]
2024 - 8 p.
- Published in : EPJ Web Conf. 295 (2024) 01008
Fulltext: PDF;
In : 26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023, pp.01008
|
|
|
2.
|
The CERN Tape Archive Beyond CERN An Open Source Data Archival System for HEP
/ Davis, Michael (CERN) ; Afonso, João (CERN) ; Bachmann, Richard (CERN) ; Bahyl, Vladimír (CERN) ; Vera, Jorge Camarero (CERN) ; Leduc, Julien (CERN) ; Cortés, Pablo Oliver (CERN) ; Rademakers, Fons (CERN) ; Wardenær, Lasse (CERN) ; Yurchenko, Volodymyr (CERN)
The CERN Tape Archive (CTA) is the successor to CASTOR and the tape backend to EOS. It was designed to meet the needs of archival storage of data from LHC Run–3 and other experimental programmes at CERN.In the wider Worldwide LHC Computing Grid (WLCG), the tape software landscape is quite heterogeneous, but we are entering a period of consolidation. [...]
2024 - 8 p.
- Published in : EPJ Web Conf. 295 (2024) 01048
Fulltext: PDF;
In : 26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023, pp.01048
|
|
|
3.
|
CERN Tape Archive Run 3 Production Experience CTA Tier-0 service performance during the start of the LHC Run 3 and the various lessons learnt
/ Leduc, Julien (CERN) ; Afonso, João (CERN) ; Bachmann, Richard (CERN) ; Bahyl, Vladimír (CERN) ; Vera, Jorge Camarero (CERN) ; Davis, Michael (CERN) ; Cortés, Pablo Oliver (CERN) ; Rademakers, Fons (CERN) ; Wardenær, Lasse (CERN) ; Yurchenko, Volodymyr (CERN)
The EOS disk + CERN Tape Archive (EOSCTA) service is CERN’s primary physics data long-term storage and archival solution for LHC Run 3. It entered production at CERN during summer 2020 and has since been serving all the LHC and non-LHC workflows involving archival to-and retrieval from tape.The CTA system is a complete redesign of the previous tape software, tape cache and tape workflows, which will need to scale to the data rate requirements of the present LHC activity period, as well as the one after. [...]
2024 - 8 p.
- Published in : EPJ Web Conf. 295 (2024) 01049
Fulltext: PDF;
In : 26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023, pp.01049
|
|
|
4.
|
ATLAS HL-LHC Demonstrators with Data Carousel: Data-on-Demand and Tape Smart Writing
/ Borodin, Misha (University of Iowa (US)) ; Klimentov, Alexei (Brookhaven National Laboratory (US)) ; Korchuganova, Tatiana (University of Pittsburgh (US)) ; Lassnig, Mario (CERN) ; Leduc, Julien (CERN) ; Maeno, Tadashi (Brookhaven National Laboratory (US)) ; Musheghyan, Haykuhi (Georg August Universitaet Goettingen (DE)) ; Zhao, Xin (Brookhaven National Laboratory (US)) ; Ressmann, Doris (FZK)
/ATLAS Collaboration
The High Luminosity upgrade to the LHC (HL-LHC) is expected to generate scientific data on the scale of multiple exabytes. To tackle this unprecedented data storage challenge, the ATLAS experiment initiated the Data Carousel project in 2018. [...]
ATL-SOFT-SLIDE-2024-546.-
Geneva : CERN, 2024 - 14 p.
Fulltext: PDF; External link: Original Communication (restricted to ATLAS)
In : 27th International Conference on Computing in High Energy & Nuclear Physics, Kraków, Pl, 19 - 25 Oct 2024
|
|
|
5.
|
|
Archive metadata
/ Leduc, Julien (speaker) (CERN)
The CTA software and service have been designed to match the Run-3 write performance requirements at WLCG Tier-0. The queuing system, coupled with a small SSD-based cache, has demonstrated its performance predictability over the first two years of the run.
This performance was achieved with FIFO scheduling, resulting in a pure temporal collocation on tapes, with only the legacy "storage class" concept to separate data into different tape pools.
As tape data is becoming "warmer" in experiment data workflows, the performance of data retrieval from tape is becoming more important. [...]
2024 - 1564.
HEP Computing; CERN Tape Archive Workshop : CTA 2024
External links: Talk details; Event details
In : CERN Tape Archive Workshop : CTA 2024
|
|
|
6.
|
|
CTA operations best practices
/ Oliver Cortes, Pablo (speaker) (CERN) ; Leduc, Julien (speaker) (CERN)
During three years in production at CERN as the WLCG Tier-0 tape storage service, the CTA software has evolved and gained various features driven by the service requirements. Discussions between developers and the operations have led to design choices that shaped the service and helped refine CTA operations best practices.
This presentations aims at clarifying the current operations and user data transfer best practices when dealing with a CTA tape system..
2024 - 1239.
HEP Computing; CERN Tape Archive Workshop : CTA 2024
External links: Talk details; Event details
In : CERN Tape Archive Workshop : CTA 2024
|
|
|
7.
|
|
|
8.
|
|
|
9.
|
|
How to enable EOS for tape
/ Leduc, Julien (speaker) (CERN)
An EOSCTA instance is an EOS instance - commonly called a tape buffer - configured with a CERN Tape Archive (CTA) back-end.
This EOS instance is entirely bandwidth oriented: it offers an SSD based tape interconnection, it can contain spinning disks if needed and it is optimized for the various tape workflows.
This talk will present how to enable EOS for tape using CTA and the Swiss horology gears in place to maximize tape hardware usage while meeting experiment workflow requirements for xrootd and HTTP protocols..
2023 - 1189.
HEP Computing; EOS 2023 Workshop
External links: Talk details; Event details
In : EOS 2023 Workshop
|
|
|
10.
|
LHC Data Storage: Preparing for the Challenges of Run-3
/ Arsuaga-Rios, Maria (CERN) ; Bahyl, Vladimír (CERN) ; Batalha, Manuel (CERN) ; Caffy, Cédric (CERN) ; Cano, Eric (CERN) ; Capitoni, Niccolo (CERN) ; Contescu, Cristian (CERN) ; Davis, Michael (CERN) ; Alvarez, David Fernandez (CERN) ; Guenther, Jaroslav (CERN) et al.
The CERN IT Storage Group ensures the symbiotic development and operations of storage and data transfer services for all CERN physics data, in particular the data generated by the four LHC experiments (ALICE, ATLAS, CMS and LHCb).In order to accomplish the objectives of the next run of the LHC (Run-3), the Storage Group has undertaken a thorough analysis of the experiments’ requirements, matching them to the appropriate storage and data transfer solutions, and undergoing a rigorous programme of testing to identify and solve any issues before the start of Run-3.In this paper, we present the main challenges presented by each of the four LHC experiments. We describe their workflows, in particular how they communicate with and use the key components provided by the Storage Group: the EOS disk storage system; its archival back-end, the CERN Tape Archive (CTA); and the File Transfer Service (FTS). [...]
2021 - 10 p.
- Published in : EPJ Web Conf. 251 (2021) 02023
Fulltext: PDF;
In : 25th International Conference on Computing in High-Energy and Nuclear Physics (CHEP), Online, Online, 17 - 21 May 2021, pp.02023
|
|