CERN Accelerating science

CMS Note
Report number	CMS-CR-2009-095
Title	Job Life Cycle Management Libraries for CMS Workflow Management Projects
Author(s)	Van Lingen, Frank (Caltech) ; Evans, Dave (Fermilab) ; Metson, Simon (Bristol U.) ; Wakefield, Stuart (Imperial Coll., London) ; Wilkinson, Rick (Caltech) ; Jackson, James (Bristol U. ; Rutherford) ; Spiga, Daniele (CERN) ; Foulkes, Stephen (Fermilab) ; Afaq, Anzar (Fermilab) ; Kuznetsov, Valentin (Cornell U., Ithaca, Phys. Dept. ) ; Vaandering, Eric (Fermilab) ; Ryu, Seangchan (Fermilab) ; Farina, Fabio (INFN, Milan) ; Codispoti, Giuseppe (Bologna U. ; INFN, Bologna) ; Cinquilli, Mattia (Perugia U. ; INFN, Perugia)
Publication	2010
Imprint	14 May 2009
Number of pages	11
In:	J. Phys.: Conf. Ser. 219 (2010) 042024
In:	17th International Conference on Computing in High Energy and Nuclear Physics, Prague, Czech Republic, 21 - 27 Mar 2009, pp.042024
DOI	10.1088/1742-6596/219/4/042024
Subject category	Detectors and Experimental Techniques
Accelerator/Facility, Experiment	CERN LHC ; CMS
Free keywords	SOFTWARE
Abstract	Scientific analysis and simulation requires the processing and generation of millions of data samples. These processing and generation tasks are often comprised of multiple smaller tasks divided over multiple (computing) sites. This paper discusses the Compact Muon Solenoid (CMS) workflow infrastructure, and specifically the Python based workflow library which is used for so called task lifecycle management. The CMS workflow infrastructure consists of three layers: high level specification of the various tasks based on input/output datasets, life cycle management of task instances derived from the high level specification and execution management. The workflow library is the result of a convergence of three CMS subprojects that respectively deal with scientific analysis, simulation and real time data aggregation from the experiment.
Copyright/License	Preprint: (License: CC-BY-4.0)

Corresponding record in: Inspire