No abstract available.
The Grid in a nutshell
The emergence and widespread adoption of Grid computing has been fueled by continued growth in both our understanding of application requirements and the sophistication of the technologies used to meet these requirements. We provide an introduction to ...
Ten actions when Grid scheduling: the user as a Grid scheduler
In this chapter we present a general architecture or plan for scheduling on a Grid. A Grid scheduler (or broker) must make resource selection decisions in an environment where it has no control over the local resources, the resources are distributed, ...
Application requirements for resource brokering in a Grid environment
We discuss the problem of resource brokering in a Grid environment from the perspective of general application needs. Starting from a illustrative scenario, these requirements are broken down into the general areas of computation, data, network, ...
Attributes for communication between Grid scheduling instances
Typically, Grid resources are subject to individual access and usage policies because they are provided by different owners. These policies are usually enforced by local management systems that maintain control of the resources. However, few Grid users ...
Security issues of Grid resource management
Secure management of Grid resources presents many challenges. This chapter will examine the security requirements that are essential to Grids and some of the software that is available to meet them. We will discuss how well these security tools have ...
Scheduling in the Grid application development software project
- Holly Dail,
- Otto Sievert,
- Fran Berman,
- Henri Casanova,
- Asim YarKhan,
- Sathish Vadhiyar,
- Jack Dongarra,
- Chuang Liu,
- Lingyun Yang,
- Dave Angulo,
- Ian Foster
Developing Grid applications is a challenging endeavor that at the moment requires both extensive labor and expertise. The Grid Application Development Software Project (GrADS) provides a system to simplify Grid application development. This system ...
Workflow management in GriPhyN
This chapter describes the work done within the NSF-funded GriPhyN project in the area of workflow management. The targeted workflows are large both in terms of the number of tasks in a given workflow and in terms of the total execution time of the ...
Grid service level agreements: Grid resource management with intermediaries
We present a reformulation of the well-known GRAM architecture based on the Service-Level Agreement (SLA) negotiation protocols defined within the Service Negotiation and Access Protocol (SNAP) framework. We illustrate how a range of local, distributed, ...
Condor and preemptive resume scheduling
Condor is a batch job system that, unlike many other scheduling systems, allows users to access both dedicated computers and computers that are not always available, perhaps because they are used as desktop computers or are not under local control. This ...
Grid resource management in legion
Grid resource management is not just about scheduling jobs on the fastest machines, but rather about scheduling all compute objects and all data objects on machines whose capabilities match the requirements, while preserving site autonomy, recognizing ...
Grid scheduling with Maui/Silver
This chapter provides an overview of the interactions of and services provided by the Maui/Silver Grid scheduling system. The Maui Scheduler provides high performance scheduling for local clusters including resource reservation, availability estimation, ...
Scheduling attributes and platform LSF
Scheduling is highly complex in the context of Grid Computing. To draw out this complexity, it makes sense to isolate and investigate key areas of the problem. Here we report on communication attributes between higher- and lower-level scheduling ...
PBS Pro: Grid computing and scheduling attributes
The PBS Pro software is a full-featured workload management and job scheduling system with capabilities that cover the entire Grid computing space: security, information, compute, and data. The security infrastructure includes user authentication, ...
Performance information services for computational Grids
Grid schedulers or resource allocators (whether they be human or automatic scheduling programs) must choose the right combination of resources from the available resource pool while the performance and availability characteristics of the individual ...
Using predicted variance for conservative on shared resources
In heterogeneous and dynamic environments, efficient execution of parallel computations can require mappings of tasks to processors with performance that is both irregular and time varying. We propose a conservative scheduling policy that uses ...
Improving resource selection and scheduling using predictions
The introduction of computational Grids has resulted in several new problems in the area of scheduling that can be addressed using predictions. The first problem is selecting where to run an application on the many resources available in a Grid. Our ...
The classads language
The Classified Advertisements (ClassAds) language facilitates the representation and participation of heterogeneous resources and customers in the resource discovery and scheduling frameworks of highly dynamic distributed environments. Although ...
Multicriteria aspects of Grid resource management
Grid resource management systems should take into consideration the application requirements and user preferences on the one hand and virtual organizations' polices on the other hand. In order to satisfy both users and resource owners, many metrics, ...
A metaheuristic approach to scheduling workflow jobs on a Grid
In this chapter we consider the problem of scheduling workflow jobs on a Grid. This problem consists in assigning Grid resources to tasks of a workflow job across multiple administrative domains in such a way that minimizes the execution time of a ...
Storage resource managers: essential components for the Grid
Storage Resource Managers (SRMs) are middleware components whose function is to provide dynamic space allocation and file management of shared storage components on the Grid. They complement Compute Resource Managers and Network Resource Managers in ...
NeST: a Grid enabled storage appliance
- John Bent,
- Venkateshwaran Venkataramani,
- Nick LeRoy,
- Alain Roy,
- Joseph Stanley,
- Andrea C. Arpaci-Dusseau,
- Remzi H. Arpaci-Dusseau,
- Miron Livny
We describe NeST, a flexible software-only storage appliance designed to meet the storage needs of the Grid. NeST has three key features that make it well-suited for deployment in a Grid environment. First, NeST provides a generic data transfer ...
Computation scheduling and data replication algorithms for data Grids
Data Grids seek to harness geographically distributed resources for large-scale data-intensive problems such as those encountered in high energy physics, bioinformatics, and other disciplines. These problems typically involve numerous, loosely coupled ...
GARA: a uniform quality of service architecture
Many Grid applications, such as interactive and collaborative environments, can benefit from guarantees for resource performance or quality of service (QoS). Although QoS mechanisms have been developed for different types of resources, they are often ...
QoS-aware service composition for large-scale peer-to-peer systems
In this chapter, we present a scalable QoS-aware service Composition framework, SpiderNet for large-scale peer-to-peer (P2P) systems. The SpiderNet framework comprises: (1) service path selection, which is responsible for selecting and composing proper ...
A peer-to-peer approach to resource location in Grid environments
Resource location (or discovery) is a fundamental service for resource-sharing environments: given desired resource attributes, the service returns locations of matching resources. Designing such a service for a Grid environment of the scale and ...
Resource management in the entropia system
Resource management for desktop Grids is particularly challenging among Grid resource management because of the heterogeneity in system, network, and sharing of resources with desktop users. Desktop Grids must support thousands to millions of computers ...
Resource management for the Triana peer-to-peer services
In this chapter we discuss the Triana problem solving environment and its distributed implementation. Triana-specific distribution mechanisms are described along with the corresponding mappings. We outline the middleware independent nature of this ...
Grid resource commercialization: economic engineering and delivery scenarios
In this chapter we consider the architectural steps needed to commercialize Grid resources as technical focus shifts towards business requirements. These requirements have been met for conventional utilities resources through commoditization, a variety ...
Trading Grid services within the UK e-science Grid
The Open Grid Services Architecture (OGSA) presents the Grid community with an opportunity to define standard service interfaces to enable the construction of an interoperable Grid infrastructure. The provision of this infrastructure has, to date, come ...
Applying economic scheduling methods to Grid environments
Scheduling becomes more difficult when resources are geographically distributed and owned by individuals with different access and cost policies. This chapter addresses the idea of applying economic models to Grid scheduling. We describe a scheduling ...
References
Cited By
- Ghosh T and Das S (2018). Job Scheduling in Computational Grid Using a Hybrid Algorithm Based on Particle Swarm Optimization and Extremal Optimization, Journal of Information Technology Research, 11:4, (72-86), Online publication date: 1-Oct-2018.
- Chmaj G and Walkowiak K (2013). A P2P computing system for overlay networks, Future Generation Computer Systems, 29:1, (242-249), Online publication date: 1-Jan-2013.
- Walkowiak K and Rak J 1+1 protection of overlay distributed computing systems Proceedings of the 12th international conference on Computational Science and Its Applications - Volume Part IV, (498-513)
- Walkowiak K, Kasprzak A, Kosowski M and Miziołek M Scheduling and capacity design in overlay computing systems Proceedings of the 12th international conference on Computational Science and Its Applications - Volume Part IV, (514-529)
- Singh S and Bawa R An efficient job scheduling algorithm for grid computing Proceedings of the International Conference on Advances in Computing and Artificial Intelligence, (216-219)
- Mehmood Shah S, Mahmood A and Oxley A Analysis and evaluation of grid scheduling algorithms using real workload traces Proceedings of the International Conference on Management of Emergent Digital EcoSystems, (234-239)
- Ali H, Saleh A, Sarhan A and Azab A (2010). Peer-to-Peer Desktop Grids Based on an Adaptive Decentralized Scheduling Mechanism, International Journal of Grid and High Performance Computing, 2:1, (1-20), Online publication date: 1-Jan-2010.
- Preve N (2010). Balanced Job Scheduling Based on Ant Algorithm for Grid Network, International Journal of Grid and High Performance Computing, 2:1, (34-50), Online publication date: 1-Jan-2010.
- Balicki J, Balicka H, Masiejczyk J and Zacniewski A Multi-criterion decision making in distrbiuted systems by quantum evolutionary algorithms Proceedings of the European conference of systems, and European conference of circuits technology and devices, and European conference of communications, and European conference on Computer science, (328-333)
- Boukerche A and Grande R Dynamic Load Balancing Using Grid Services for HLA-Based Simulations on Large-Scale Distributed Systems Proceedings of the 2009 13th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications, (175-183)
- Yau S, Karamcheti V, Zorin D, Damevski K and Parker S Application-aware management of parallel simulation collections Proceedings of the 14th ACM SIGPLAN symposium on Principles and practice of parallel programming, (35-44)
- Yau S, Karamcheti V, Zorin D, Damevski K and Parker S (2009). Application-aware management of parallel simulation collections, ACM SIGPLAN Notices, 44:4, (35-44), Online publication date: 14-Feb-2009.
- Korkhov V, Moscicki J and Krzhizhanovskaya V (2009). Dynamic workload balancing of parallel applications with user-level scheduling on the Grid, Future Generation Computer Systems, 25:1, (28-34), Online publication date: 1-Jan-2009.
- Abdelkader K and Broeckhove J (2009). Pricing computational resources in a dynamic grid, International Journal of Grid and Utility Computing, 1:3, (205-215), Online publication date: 1-Aug-2009.
- Balicki J (2009). Some numerical experiments on multi-criterion tabu programming for finding Pareto-optimal solutions, WSEAS TRANSACTIONS on SYSTEMS, 8:2, (241-250), Online publication date: 1-Feb-2009.
- Balicki J An adaptive quantum-based multiobjective evolutionary algorithm for efficient task assignment in distributed systems Proceedings of the WSEAES 13th international conference on Computers, (417-422)
- Walkowiak K and Woźniak M Modeling of network computing systems for decision tree induction tasks Proceedings of the 10th international conference on Intelligent data engineering and automated learning, (759-766)
- Balicki J Multi-criterion decision making by artificial intelligence techniques Proceedings of the 8th WSEAS international conference on Artificial intelligence, knowledge engineering and data bases, (319-324)
- Schlegel T, Kowalczyk R and Vo Q Decentralized Co-allocation of Interrelated Resources in Dynamic Environments Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 02, (104-108)
- v. d. Kuijl A, Emmerich M and Li H A novel multi-objective optimization scheme for grid resource allocation Proceedings of the 6th international workshop on Middleware for grid computing, (1-6)
- Balicki J Multi-criterion tabu programming for pareto-optimal task assignment in distributed computer systems Proceedings of the 12th WSEAS international conference on Computers, (142-147)
- Gallardo A, Díaz De Cerio L and Sanjeevan K Self-configuring Resource Discovery on a Hypercube Grid Overlay Proceedings of the 14th international Euro-Par conference on Parallel Processing, (510-519)
- Scorsatto G and Melo A GrAMoS Proceedings of the 14th international Euro-Par conference on Parallel Processing, (534-543)
- Chmaj G and Walkowiak K Heuristic Algorithm for Optimization of P2P-Based Public-Resource Computing Systems Proceedings of the 5th International Conference on Distributed Computing and Internet Technology, (180-187)
- Stankovski V, Swain M, Kravtsov V, Niessen T, Wegener D, Kindermann J and Dubitzky W (2008). Grid-enabling data mining applications with DataMiningGrid, Future Generation Computer Systems, 24:4, (259-279), Online publication date: 1-Apr-2008.
- Li H and Muskulus M (2007). Analysis and modeling of job arrivals in a production grid, ACM SIGMETRICS Performance Evaluation Review, 34:4, (59-70), Online publication date: 1-Mar-2007.
- Shang L, Wang Z, Zhou X, Huang X and Cheng Y TM-DG Proceedings of the 2007 symposium on Component and framework technology in high-performance and scientific computing, (59-66)
- Hasselmeyer P Removing the need for state dissemination in grid resource brokering Proceedings of the 5th international workshop on Middleware for grid computing: held at the ACM/IFIP/USENIX 8th International Middleware Conference, (1-6)
- Ellert M, Grønager M, Konstantinov A, Kónya B, Lindemann J, Livenson I, Nielsen J, Niinimäki M, Smirnova O and Wäänänen A (2018). Advanced resource connector middleware for lightweight computational Grids, Future Generation Computer Systems, 23:2, (219-240), Online publication date: 1-Feb-2007.
- Tan Z and Gurd J Market-based grid resource allocation using a stable continuous double auction Proceedings of the 8th IEEE/ACM International Conference on Grid Computing, (283-290)
- Magaña E, Lefevre L and Serrat J Autonomic management architecture for flexible grid services deployment based on policies Proceedings of the 20th international conference on Architecture of computing systems, (157-170)
- Liang Y, Fan J, Meng D and Di R A strategy-proof combinatorial auction-based grid resource allocation system Proceedings of the 7th international conference on Algorithms and architectures for parallel processing, (254-266)
- Magaña E, Lefevre L, Hasan M and Serrat J SNMP-based monitoring agents and heuristic scheduling for large-scale grids Proceedings of the 2007 OTM confederated international conference on On the move to meaningful internet systems: CoopIS, DOA, ODBASE, GADA, and IS - Volume Part II, (1367-1384)
- Dethier G, Briquet C, Marchot P and De Marneffe P A grid-enabled Lattice-Boltzmann-based modelling system Proceedings of the 7th international conference on Parallel processing and applied mathematics, (1275-1284)
- Gréhant X, Pernet O, Jarp S, Demeure I and Toft P Xen management with SmartFrog Proceedings of the 2007 conference on Parallel processing, (205-213)
- Krzhizhanovskaya V and Korkhov V Dynamic load balancing of black-box applications with a resource selection mechanism on heterogeneous resources of the grid Proceedings of the 9th international conference on Parallel Computing Technologies, (245-260)
- Allenotor D and Thulasiram R A grid resources valuation model using fuzzy real option Proceedings of the 5th international conference on Parallel and Distributed Processing and Applications, (622-632)
- Briquet C and de Marneffe P Grid resource negotiation Proceedings of the 24th IASTED international conference on Parallel and distributed computing and networks, (17-22)
- Boukerche A, Sousa M and de Melo A A mltiple task allocation frame work for biological seqence comparision in a grid environment Proceedings of the 20th international conference on Parallel and distributed processing, (247-247)
- Yau S, Grinspun E, Karamcheti V and Zorin D Sim-X Proceedings of the 20th international conference on Parallel and distributed processing, (137-137)
- Jelasity M, Babaoglu O, Laddaga R, Nagpal R, Zambonelli F, Sirer E, Chaouchi H and Smirnov M (2006). Interdisciplinary Research, IEEE Intelligent Systems, 21:2, (50-58), Online publication date: 1-Mar-2006.
- Li H, Muskulus M and Wolters L Modeling job arrivals in a data-intensive grid Proceedings of the 12th international conference on Job scheduling strategies for parallel processing, (210-231)
- Stucky K, Jakob W, Quinte A and Süß W Solving scheduling problems in grid resource management using an evolutionary algorithm Proceedings of the 2006 Confederated international conference on On the Move to Meaningful Internet Systems: CoopIS, DOA, GADA, and ODBASE - Volume Part II, (1252-1262)
- Balicki J Negative selection with ranking procedure in tabu-based multi-criterion evolutionary algorithm for task assignment Proceedings of the 6th international conference on Computational Science - Volume Part III, (863-870)
- Placek M and Buyya R Storage exchange Proceedings of the 12th international conference on Parallel Processing, (425-436)
- Anglano C, Brevik J, Canonico M, Nurmi D and Wolski R Fault-aware scheduling for Bag-of-Tasks applications on Desktop Grids Proceedings of the 7th IEEE/ACM International Conference on Grid Computing, (56-63)
- Wieczorek M, Prodan R and Fahringer T (2005). Scheduling of scientific workflows in the ASKALON grid environment, ACM SIGMOD Record, 34:3, (56-62), Online publication date: 1-Sep-2005.
- Vargas P, de Castro Dutra I, Dalto do Nascimento V, Santos L, da Silva L, Geyer C and Schulze B Hierarchical submission in a Grid environment Proceedings of the 3rd international workshop on Middleware for grid computing, (1-6)
- Das A and Grosu D Combinatorial Auction-Based Protocols for Resource Allocation in Grids Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Workshop 13 - Volume 14
- Weigang L, Dib M and de Melo A Method to balance the communication among multi-agents in real time traffic synchronization Proceedings of the Second international conference on Fuzzy Systems and Knowledge Discovery - Volume Part I, (1053-1062)
- Rodero I, Corbalán J, Badia R and Labarta J eNANOS grid resource broker Proceedings of the 2005 European conference on Advances in Grid Computing, (111-121)
- Wieczorek M, Prodan R and Fahringer T Comparison of workflow scheduling strategies on the grid Proceedings of the 6th international conference on Parallel Processing and Applied Mathematics, (792-800)
- Kwiatkowski J, Karwaczyński P and Pawlik M Dependable information service for distributed systems Proceedings of the 6th international conference on Parallel Processing and Applied Mathematics, (1010-1017)
- Talwar V, Agarwalla B, Basu S, Kumar R and Nahrstedt K Architecture for resource allocation services supporting interactive remote desktop sessions in utility grids Proceedings of the 2nd workshop on Middleware for grid computing, (23-28)
- Modahl M, Agarwalla B, Abowd G, Ramachandran U and Saponas T Toward a standard ubiquitous computing framework Proceedings of the 2nd workshop on Middleware for pervasive and ad-hoc computing, (135-139)
- Weigang L, Pinheiro Dib M and Cardoso D Grid Service Agents for Real Time Traffic Synchronization Proceedings of the 2004 IEEE/WIC/ACM International Conference on Web Intelligence, (619-623)
Index Terms
- Grid resource management: state of the art and future trends
Reviews
Art Sedighi
The editors have done a great job, putting together a text that details current trends, products, and research in grid computing, from start to finish. The text starts with a handful of papers that offer an overview of grid computing, and present the general topic of resource management in grid environments. Resource management is explained as "the ability to discover, allocate, negotiate, monitor, and manage the use of network-accessible capabilities in order to achieve various end-to-end or global qualities of service." Resource management software must achieve all of these tasks, in order to efficiently schedule jobs that best use the available resources of a grid infrastructure. The overview of grid leads to the concept of grid schedulers and their attributes. Essentially, the authors start out with the problem statements (in the first part of the book), and the rest of the text consists of works by various researchers in academia or industry who are trying to find solutions using different techniques. Platform LSF, the portable batch system (PBS), Legion, Condor, and Maui are the main industry-strength grid resource management software applications that are covered. The authors cover their products in detail, which essentially means that they answer all of the questions raised in the first section of the book. Emphasis is given to the commercial aspects of grid resource managers, since there are a number of them available that closely follow the latest research efforts in academia. As you may imagine, resource scheduling is a predictive task. Current research focuses on coming up with algorithms, methodologies, and techniques, taken from machine learning, artificial intelligence (AI), and neural networks, that can be applied to, and used to address, the problem of resource scheduling. Some of these techniques are not yet ready for industry, but projects exist, such as the Network Weather Service or the grid analysis and display system (GrADS) project, that take advantage of current research efforts in this field. Unlike the section on available software packages from industry, where many deployments exist, the authors focusing on the latest research try to take a more direct, case study-driven approach to their work. It is rather interesting to see how small the gap really is between industry-strength software applications for resource management and the current state of research by the top researchers. This shows how new this technology really is. In addition to the scheduling of computer resources, other important issues (data scheduling, replication, movement of data, and so on) are shown to be an essential part of grid resource management. The authors delve into many aspects of storage and data file management across a grid. Interesting edge cases, such as scenarios where multiple administrative environments are involved, are also depicted. Quality of service (QoS) for grid computing has a special meaning, since it no longer applies only to network resources. Computing, data, and network resources together need to be managed, and there needs to be a mechanism that provides a quantifiable way of dictating QoS across all three domains. Scheduling systems, commercial or academic, need to take QoS guarantees into account when scheduling tasks across resources and administrative domains. The concept of QoS and data scheduling is further complicated when talking about peer-to-peer networks, where dynamism is one of the key features. In these networks, deterministic scheduling becomes more difficult, and thus meeting QoS guarantees becomes even more complex. The papers presented in this text represent the latest research on resource management for grid computing. Resource management continues to be a difficult problem to solve, and researchers in this field have taken numerous approaches to the problem. What technique, if any, prevails remains to be seen, but this text certainly puts the latest information at your fingertips, and allows you to come to your own conclusion about what methodology best solves your specific problem. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Recommendations
The Grid Resource Broker workflow engine
2nd International Workshop on Workflow Management and Applications in Grid Environments (WaGe2007)Increasingly, complex scientific applications are structured in terms of workflows. These applications are usually computationally and-or data intensive and thus are well suited for execution in grid environments. Distributed, geographically spread ...
The Organization and Management of Grid Infrastructures
Grid computing technology has become fundamental to e-Science. As the virtual organizations established by scientific communities progress from testing their applications to more routine usage, maintaining reliable and adaptive grid infrastructures ...