| ATLAS Note | |
| Report number | ATL-SOFT-PROC-2019-003 |
| Title | The ATLAS EventIndex and its evolution towards Run 3 |
| Author(s) |
Villaplana Perez, Miguel (INFN Milano and Universita' di Milano, Dipartimento di Fisica) ; Alexandrov, Evgeny (Joint Institute for Nuclear Research) ; Aleksandrov, Igor (Joint Institute for Nuclear Research) ; Baranowski, Zbigniew (CERN) ; Barberis, Dario (INFN Genova and Universita' di Genova, Dipartimento di Fisica) ; Dimitrov, Gancho (European Laboratory for Particle Physics, CERN) ; Fernandez Casani, Alvaro (Instituto de Fisica Corpuscular (IFIC), Centro Mixto Universidad de Valencia - CSIC) ; Gallas, Elizabeth (University of Oxford, Particle Physics) ; Garcia Montoro, Carlos (Instituto de Fisica Corpuscular (IFIC), Centro Mixto Universidad de Valencia - CSIC) ; Gonzalez de la Hoz, Santiago (Instituto de Fisica Corpuscular (IFIC), Centro Mixto Universidad de Valencia - CSIC) ; Hrivnac, Julius (LAL, Univ. Paris-Sud, IN2P3/CNRS, Universite Paris-Saclay) ; Iakovlev, Alexander (Joint Institute for Nuclear Research) ; Kazymov, Andrei (Joint Institute for Nuclear Research) ; Mineev, Mikhail (Joint Institute for Nuclear Research) ; Prokoshin, Fedor (Joint Institute for Nuclear Research) ; Rybkin, Grigori (LAL, Univ. Paris-Sud, IN2P3/CNRS, Universite Paris-Saclay) ; Sánchez, Javier (Instituto de Fisica Corpuscular (IFIC), Centro Mixto Universidad de Valencia - CSIC) ; Salt, José (Instituto de Fisica Corpuscular (IFIC), Centro Mixto Universidad de Valencia - CSIC) ; Vasileva, Petya Tsvetanova (The University of Texas at Arlington) |
| Corporate Author(s) | The ATLAS collaboration |
| Publication | 2020 |
| Imprint | 19 May 2019 |
| Number of pages | 6 |
| In: | J. Phys.: Conf. Ser. 1525 (2020) 012056 |
| In: | 19th International Workshop on Advanced Computing and Analysis Techniques in Physics Research, Saas Fee, Switzerland, 11 - 15 Mar 2019, pp.012056 |
| DOI | 10.1088/1742-6596/1525/1/012056 |
| Subject category | Particle Physics - Experiment |
| Accelerator/Facility, Experiment | CERN LHC ; ATLAS |
| Free keywords | Scientific Computing ; BigData ; Hadoop ; Apache Phoenix ; EventIndex |
| Abstract | The ATLAS experiment produced so far hundreds of petabytes of data and expects to have one order of magnitude more in the future. This data are spread among hundreds of computing Grid sites around the world. The EventIndex is the complete catalogue of all ATLAS events, real and simulated, keeping the references to all permanent files that contain a given event in any processing stage. It provides the means to select and access event data in the ATLAS distributed storage system, and provides support for completeness and consistency checks and trigger and offline selection overlap studies. The EventIndex employs various data handling technologies like Hadoop and Oracle databases, and is integrated with other systems of the ATLAS distributed computing infrastructure, including those for data, metadata, and production management. The project is in operation since the start of LHC Run 2 in 2015, and is in permanent development in order to fit the production and analysis demands and follow technology evolutions. The main data store in Hadoop, based on MapFiles and HBase, has worked well during Run 2 but new solutions are explored for the future. This paper reports on the current system performance and on the studies of a new data storage prototype that can carry the EventIndex through Run 3. |
| Related document | Slides ATL-SOFT-SLIDE-2019-075 |
| Copyright/License | publication: (License: CC-BY-3.0) |
Corresponding record in: Inspire
Record created 2019-05-19, last modified 2022-01-14