Production grids integrate today thousands of resources into e-Science platforms. However, the current practice of running yearly tens of millions of single-resource, long-running grid jobs with few fault tolerance capabilities is hampered by the highly dynamic grid resource availability. In additional to resource failures, grids introduce a new vector of resource availability dynamics: the resource sharing policy established by the resource owners. As a result, the availability-aware grid resourcemanagement is a challenging problemfor today’s researchers. To address this problem, we present in this work GriS-Prophet, an integrated system for resource availability monitoring, analysis, and prediction. Using GriS-Prophet’s analysis tools on a long-term availability trace from the Austrian Grid, we characterize the grid resource availability for three resource availability policies. Notably, we show that the three policies lead to very different capabilities for running the typical grid workloads efficiently. We introduce a new resource availability predictor based on Bayesian inference. Last but not least, using GriS-Prophet’s prediction tools we achieve an accuracy of more than 90%; and 75%; in our instance and duration availability predictions respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
A. Acharya, G. Edjlali, and J. H. Saltz. The utility of exploiting idle workstations for parallel computation. In SIGMETRICS, pages 225-236, 1997.
R. Bhagwan, S. Savage, and G. M. Voelker. Understanding availability. In IPTPS, pages 256-267, 2003.
W. M. Bolstad. Introduction to Bayesian Statistics. Aug. 2007.
Derrick Kondo et al. Characterizing resource availability in enterprise desktop grids. Future Generation Comp. Syst., 23(7):888-903, 2007.
P. A. Dinda. A prediction-based real-time scheduling advisor. In IPDPS. IEEE Computer Society, 2002.
EGEE Team, LCG. [Online] http://lcg.web.cern.ch/, 2007.
S. Fu and C.-Z. Xu. Exploring event correlation for failure prediction in coalitions of clusters. In SC. ACM, 2007.
A. Iosup, C. Dumitrescu, D. H. J. Epema, H. Li, and L. Wolters. How are real grids used? the analysis of four grid traces and its implications. In GRID, pages 262-269. IEEE, 2006.
A. Iosup, M. Jan, O. Sonmez, and D. Epema. On the dynamic resources availability in grids. In Grid 2007, Austin, TX, USA, September 19-21.
J. W. Mickens and B. D. Noble. Exploiting availability prediction in distributed systems. In NSDI. USENIX, 2006.
R. Wolski et al. Automatic methods for predicting machine availability in desktop grid and peer-to-peer systems. In CCGRID ’04.
Ramendra et al. Critical event prediction for proactive management in large-scale com- puter clusters. In KDD, pages 426-435, 2003.
X. Ren, S. Lee, R. Eigenmann, and S. Baghci. Resource availability prediction in finegrained cycle sharing systems. In HPDC, 2006.
B. Rood and M. J. Lewis. Multi-state grid resource availability characterization. In Grid 2007, Austin, TX, September 17-19,.
B. Schroeder and G. A. Gibson. A large-scale study of failures in high-performance computing systems. In DSN, pages 249-258. IEEE Computer Society, 2006.
D. Tang and R. K. Iyer. Dependability measurement and modeling of a multicomputer system. IEEE Trans. Comput., 42(1):62-75, 1993.
The Austrian Grid Consortium. [Online] http://www.austriangrid.at, 2007.
The TeraGrid Project. [Online] http://www.teragrid.org/, 2007.
R. Vilalta, C. Apté, J. L. Hellerstein, S. Ma, and S. M. Weiss. Predictive algorithms in the management of computer systems. IBM Systems Journal, 41(3):461-474, 2002.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Nadeem, F., Prodan, R., Fahringer, T., Iosup, A. (2008). A Framework For Resource Availability Characterization And Online Prediction In The Grids. In: Gorlatch, S., Fragopoulou, P., Priol, T. (eds) Grid Computing. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-09457-1_18
Download citation
DOI: https://doi.org/10.1007/978-0-387-09457-1_18
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-09456-4
Online ISBN: 978-0-387-09457-1
eBook Packages: Computer ScienceComputer Science (R0)