|
1.
|
EOS architectural evolution and strategic development directions
/ Bitzes, Georgios (CERN) ; Luchetti, Fabio (CERN) ; Manzi, Andrea (CERN) ; Patrascoiu, Mihai (CERN) ; Peters, Andreas Joachim (CERN) ; Simon, Michal Kamil (CERN) ; Sindrilaru, Elvin Alin (CERN)
EOS [1] is the main storage system at CERN providing hundreds of PB of capacity to both physics experiments and also regular users of the CERN infrastructure. Since its first deployment in 2010, EOS has evolved and adapted to the challenges posed by ever-increasing requirements for storage capacity, user-friendly POSIX-like interactive experience and new paradigms like collaborative applications along with sync and share capabilities.Overcoming these challenges at various levels of the software stack meant coming up with a new architecture for the namespace subsystem, completely redesigning the EOS FUSE module and adapting the rest of the components like draining, LRU engine, file system consistency check and others, to ensure a stable and predictable performance. [...]
2020 - 10 p.
- Published in : EPJ Web Conf. 245 (2020) 04009
Fulltext from publisher: PDF;
In : 24th International Conference on Computing in High Energy and Nuclear Physics, Adelaide, Australia, 4 - 8 Nov 2019, pp.04009
|
|
|
2.
|
Code health in EOS: Improving test infrastructure and overall service quality
/ Sindrilaru, Elvin Alin (CERN) ; Bitzes, Georgios (CERN) ; Luchetti, Fabio (CERN) ; Patrascoiu, Mihai (CERN)
During the last few years, the EOS distributed storage system at CERN has seen a steady increase in use, both in terms of traffic volume as well as sheer amount of stored data. This has brought the unwelcome side effect of stretching the EOS software stack to its design constraints, resulting in frequent user-facing issues and occasional downtime of critical services. [...]
2020 - 7 p.
- Published in : EPJ Web Conf. 245 (2020) 05027
Fulltext: PDF;
In : 24th International Conference on Computing in High Energy and Nuclear Physics, Adelaide, Australia, 4 - 8 Nov 2019, pp.05027
|
|
|
3.
|
|
After the hangover: QuarkDB and the new namespace
/ Bitzes, Georgios (speaker) (CERN)
In this talk we give a brief overview of the successful migration to the new namespace. Practically all EOS instances at CERN are currently on QuarkDB, the new namespace is officially *boring technology*, and *MGM boot time* a distant memory.
We will also discuss future plans and ideas to further improve scalability and performance of the namespace in particular with respect to locking, planned end-of-support for in-memory legacy namespace, and all miscellaneous namespace-related news..
2020 - 1356.
HEP Computing; EOS workshop
External links: Talk details; Event details
In : EOS workshop
|
|
|
4.
|
Scaling the EOS namespace - new developments, and performance optimizations
/ Bitzes, Georgios (CERN) ; Sindrilaru, Elvin Alin (CERN) ; Peters, Andreas Joachim (CERN)
EOS is the distributed storage solution being developed and deployed at CERN with the primary goal of fulfilling the data needs of the LHC and its various experiments. Being in production since 2011, EOS currently manages around 256 petabytes of raw disk space and 3.4 billion files across several instances. [...]
2019 - 8 p.
- Published in : EPJ Web Conf. 214 (2019) 04019
Fulltext from publisher: PDF;
In : 23rd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2018, Sofia, Bulgaria, 9 - 13 Jul 2018, pp.04019
|
|
|
5.
|
A milestone for DPM (Disk Pool Manager)
/ Furano, Fabrizio (CERN) ; Keeble, Oliver (CERN) ; Manzi, Andrea (CERN) ; Bitzes, Georgios (CERN)
The DPM (Disk Pool Manager) system is a multiprotocol scalable technology for Grid storage that supports about 130 sites for a total of about 90 Petabytes online. The system has recently completed the development phase that had been announced in the past years, which consolidates its core component (DOME: Disk Operations Management Engine) as a full-featured high performance engine that can also be operated with standard Web clients and uses a fully documented REST-based protocol. [...]
2019 - 8 p.
- Published in : EPJ Web Conf. 214 (2019) 04018
Fulltext from publisher: PDF;
In : 23rd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2018, Sofia, Bulgaria, 9 - 13 Jul 2018, pp.04018
|
|
|
6.
|
Testing of complex, large-scale distributed storage systems: a CERN disk storage case study
/ Makai, Jozsef (CERN) ; Peters, Andreas Joachim (CERN) ; Bitzes, Georgios (CERN) ; Sindrilaru, Elvin Alin (CERN) ; Simon, Michal Kamil (CERN) ; Manzi, Andrea (CERN)
Complex, large-scale distributed systems are frequently used to solve extraordinary computing, storage and other problems. However, the development of these systems usually requires working with several software components, maintaining and improving a large codebase and also providing a collaborative environment for many developers working together. [...]
2019 - 7 p.
- Published in : EPJ Web Conf. 214 (2019) 05008
Fulltext from publisher: PDF;
In : 23rd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2018, Sofia, Bulgaria, 9 - 13 Jul 2018, pp.05008
|
|
|
7.
|
|
|
8.
|
Scaling the EOS namespace
/ Peters, Andreas J (CERN) ; Sindrilaru, Elvin A (CERN) ; Bitzes, Georgios (CERN)
EOS is the distributed storage system being developed at CERN with the aim of fulfilling a wide range of data storage needs, ranging from physics data to user home directories. Being in production since 2011, EOS currently manages around 224 petabytes of disk space and 1.4 billion files across several instances. [...]
2017
In : ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Frankfurt, Germany, 18 - 22 Jun 2017, pp.731-740
|
|
|
9.
|
Making the most of cloud storage - a toolkit for exploitation by WLCG experiments
/ Ayllon, Alejandro Alvarez (CERN) ; Arsuaga Rios, Maria (CERN) ; Bitzes, Georgios (CERN) ; Furano, Fabrizio (CERN) ; Keeble, Oliver (CERN) ; Manzi, Andrea (CERN)
Understanding how cloud storage can be effectively used, either standalone or in support of its associated compute, is now an important consideration for WLCG. We report on a suite of extensions to familiar tools targeted at enabling the integration of cloud object stores into traditional grid infrastructures and workflows. [...]
2017 - 8 p.
- Published in : J. Phys.: Conf. Ser. 898 (2017) 062027
Fulltext: PDF;
In : 22nd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2016, San Francisco, Usa, 10 - 14 Oct 2016, pp.062027
|
|
|
10.
|
|
Distributed consensus and fault tolerance - Lecture 2
/ Bitzes, Georgios (speaker) (CERN)
In a world where clusters with thousands of nodes are becoming commonplace, we are often faced with the task of having them coordinate and share state. As the number of machines goes up, so does the probability that something goes wrong: a node could temporarily lose connectivity, crash because of some race condition, or have its hard drive fail.
What are the challenges when designing fault-tolerant distributed systems, where a cluster is able to survive the loss of individual nodes? In this lecture, we will discuss some basics on this topic (consistency models, CAP theorem, failure modes, byzantine faults), detail the raft consensus algorithm, and showcase an interesting example of a highly resilient distributed system, bitcoin..
2017 - 3703.
inverted CSC; Inverted CERN School of Computing 2017
External links: Talk details; Event details
In : Inverted CERN School of Computing 2017
|
|