Abstract
In 2007, the most challenging high energy physics experiment ever, the Large Hardon Collider(LHC), at CERN, will produce a sustained stream of data in the order of 300MB/sec, equivalent to a stack of CDs as high as the Eiffel Tower once per week. This data is, while produced, distributed and persistently stored at several dozens of sites around the world, building the LHC data grid. The destination sites are expected to provide the necessary middle-ware, so called Storage Elements, offering standard protocols to receive the data and to store it at the site specific Storage Systems. A major player in the set of Storage Elements is the dCache/SRM system. dCache/SRM has proven to be capable of managing the storage and exchange of several hundreds of terabytes of data, transparently distributed among dozens of disk storage nodes. One of the key design features of the dCache is that although the location and multiplicity of the data is autonomously determined by the system, based on configuration, cpu load and disk space, the name space is uniquely represented within a single file system tree. The system has shown to significantly improve the efficiency of connected tape storage systems, by caching, ’gather & flush’ and scheduled staging techniques. Furthermore, it optimizes the throughput to and from data clients as well as smoothing the load of the connected disk storage nodes by dynamically replicating datasets on the detection of load hot spots. The system is tolerant against failures of its data servers which enables administrators to go for commodity disk storage components. Access to the data is provided by various standard protocols. Furthermore the software is coming with an implementation of the Storage Resource Manager protocol (SRM), which is evolving to an open standard for grid middleware to communicate with site specific storage fabrics.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
DESY: http://www.desy.de
FERMI: http://www.fnal.gov
CERN: http://www.cern.ch
Rutherford Appleton Laboratory: http://www.cclrc.ac.uk/
Large Hadron Collider: http://lhc.web.cern.ch/lhc/
LHC Computing Grid : http://lcg.web.cern.ch/LCG/
Fermi Enstore, http://www.fnal.gov/docs/products/enstore/
High Performance Storage System: http://www.hpss-collaboration.org/hpss/
Tivoli Storage Manager : http://www-306.ibm.com/software/tivoli/products/storage-mgr/
CASTOR Storage Manager: http://castor.web.cern.ch/castor/
dCache Documentation: http://www.dcache.org
dCache, the Book: http://www.dcache.org/manuals/Book
GsiFtp http://www.globus.org/datagrid/deliverables/gsiftp-tools.html
Secure Ftp : http://www.ietf.org/rfc/rfc2228.txt
Fermi CDF Experiment : http://www-cdf.fnal.gov
GridKA : http://www.gridka.de/
Cern CMS Experiment : http://cmsinfo.cern.ch
Grid GFAL http://lcg.web.cern.ch/LCG/peb/GTA/GTA-ES/Grid-File-AccessDesign-v1.0.doc
D-Grid, The German e-science program : http://www.d-grid.de
Fuhrmann, P., et al.: dCache, the Upgrade. In: CHEP 2006, Mumbai, India (Spring 2006)
Field, L., et al.: Grid Deployment Experiences: The path to a production quality LDAP based grid information system. In: CHEP 2006, Mumbai, India (Spring 2006)
Mkrtchyan, T., et al.: Chimera. In: CHEP 2006, Mumbai, India (Spring 2006)
Schley, L., Radicke, M., et al.: A Computational and Data Scheduling Architecture for HEP Application. In: CHEP 2006, Mumbai, India (Spring 2006)
Kulyavtsev, A., et al.: Resilient dCache: Replicating Files for Integrity and Availability. In: CHEP 2006, Mumbai, India (Spring 2006)
Perelmutov, T., et al.: Enabling Grid features in dCache. In: CHEP 2006, Mumbai, India (Spring 2006)
Rana, A.S., et al.: gPLAZMA: Introducing RBAC Security in dCache . In: CHEP 2006, Mumbai, India (Spring 2006)
Fuhrmann, P., et al.: The TSM in the LHC Grid World. In: TSM Symposium, Oxford, UK (September 2005)
Fuhrmann, P.: dCache, the commodity cache. Spring 2004. In: Twelfth NASA Goddard and Twenty First IEEE Conference on Mass Storage Systems and Technologies, Washington DC, USA (2004)
Perelmutov, T.: Storage Resource Managers by CMS,LCG. Spring 2004. In: Twelfth NASA Goddard and Twenty First IEEE Conference on Mass Storage Systems and Technologies. Washington DC (2004)
Ernst, M., et al.: Managed Data Storage and Data Access Services for Data Grids. In: CHEP 2004, Interlaken, Switzerland (September 2004)
Mkrtchyan, T., et al.: Chimera, the commodity namespace service. In: CHEP 2004, Interlaken, Switzerland (September 2004)
Fuhrmann, P., et al.: dCache, LCG SE and enhanced use cases. In: CHEP 2004, Interlaken, Switzerland (September 2004)
Ernst, M., Fuhrmann, P., et al.: dCache. In: CHEP03, San Diego, USA (March 2003)
Fuhrmann, P., et al.: dCache. In: CHEP 2001, Bejing, China (September 2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fuhrmann, P., Gülzow, V. (2006). dCache, Storage System for the Future. In: Nagel, W.E., Walter, W.V., Lehner, W. (eds) Euro-Par 2006 Parallel Processing. Euro-Par 2006. Lecture Notes in Computer Science, vol 4128. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11823285_116
Download citation
DOI: https://doi.org/10.1007/11823285_116
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37783-2
Online ISBN: 978-3-540-37784-9
eBook Packages: Computer ScienceComputer Science (R0)