10cs845 Unit 4
10cs845 Unit 4
10cs845 Unit 4
Prepared by M .Chandana
Department of CSE
Grid: Resource-Sharing
Environment
Users:
1000s from 10s
institutions
Well-established
communities
Resources:
Computers, data,
instruments, storage,
applications
Owned/administered by
institutions
Functionality &
infrastructure
Grids
Large scale
Weaker trust assumptions
Ease of integration
No centralized authority
Intermittent resource/user participation
Diversity in:
Shared resources
Sharing characteristics
P2P
On Death, Taxes, and the Convergence of Grid and P2P Systems, Foster and Iamnitchi,
IPTPS03
Grid: Definitions
Definition 1: Infrastructure that provides
dependable, consistent, pervasive, and
inexpensive access to high-end computational
capabilities (1998)
How It Started
While helping to build/integrate a diverse
range of distributed applications, the same
problems kept showing up over and over
again.
Too hard to keep track of authentication data
(ID/password) across institutions
Too hard to monitor system and application
status across institutions
Too many ways to submit jobs
Too many ways to store & access files and data
Too many ways to keep track of data
Too easy to leave dangling resources lying
around (robustness)
grid architecture
in a nutshell
Forget Homogeneity!
Trying to force
homogeneity on
users is futile.
Everyone has their
own preferences,
sometimes even
dogma.
The Internet
provides the
model
Compute
Server
Simulation
Tool
Web
Portal
Registration
Service
Data
Viewer
Tool
Chat
Tool
Credential
Repository
Telepresence
Monitor
Application services
organize VOs & enable
access to other services
Camera
Camera
Database
service
Data
Catalog
Database
service
Database
service
Certificate
authority
Users work
with client
applications
Compute
Server
Collective services
aggregate &/or
virtualize resources
Resources implement
standard access &
management interfaces
Web
Browser
Web
Portal
Application
Developer
10
Off the
Shelf
12
Globus
Toolkit
Grid
Community
Compute
Server
Compute
Server
Registration
Service
Data
Viewer
Tool
Chat
Tool
Credential
Repository
Application services
organize VOs & enable
access to other services
Camera
Telepresence
Monitor
Data
Catalog
Certificate
authority
Users work
with client
applications
Collective services
aggregate &/or
virtualize resources
Camera
C
Database
service
Database
service
Database
service
Resources implement
standard access &
management interfaces
Compute
GRAM Server
Globus
Simulation
Tool
Web
Browser
Application
Developer
Off the
Shelf
Globus Toolkit
Grid
Community
Globus Index
Service
CHEF
Data
Viewer
Tool
CHEF Chat
Teamlet
MyProxy
Telepresence
Monitor
Application services
organize VOs & enable
access to other services
Camera
Camera
Database
DAI service
Globus
Globus
MCS/RLS
Database
DAI service
Globus
Database
DAI service
Globus
Certificate
Authority
Users work
with client
applications
Compute
GRAM Server
Globus
Collective services
aggregate &/or
virtualize resources
Resources implement
standard access &
management interfaces
G
T
3
G
T
2
Delegation
Service
Python WS Core
[contribution]
C WS Core
Community
OGSA-DAI
Authorization
[Tech Preview]
Service
WS
Authentication
Authorization
Pre-WS
Authentication
Authorization
G
T
3
G
T
4
Community
Scheduler
Framework
[contribution]
Web Services
Components
Reliable
File
Transfer
Grid
Monitoring
Resource
& Discovery
Allocation Mgmt
System
(WS GRAM)
(MDS4)
Java WS Core
GridFTP
Grid
Monitoring
Resource
& Discovery
Allocation Mgmt
System
(Pre-WS GRAM)
(MDS2)
C Common
Libraries
Replica
Location
Service
Components
XIO
Credential
Management
Security
Data
Management
Execution
Management
Information
Services
Non-WS
Common
Runtime
In reality:
Different history
Cloud computing as utility computing (1966 paper)
Outline
Why Now?
Experience with very large
datacenters
Unprecedented economies of scale
Other factors
Pervasive broadband Internet
Fast x86 virtualization
Pay-as-you-go billing model
Standard software stack
21
Spectrum of Clouds
Instruction Set VM (Amazon EC2, 3Tera)
Bytecode VM (Microsoft Azure)
Framework VM
Google AppEngine, Force.com
Lower-level,
Less management
EC2
Higher-level,
More management
Azure
AppEngine Force.com
22
23
Capacity
Capacity
Demand
Demand
Time
Time
Capacity
Unused resources
Demand
Time
Capacity
Demand
Capacity
Demand
2
1
Time (days)
2
1
Time (days)
Lost revenue
3
Capacity
Demand
2
1
Time (days)
Lost users
26
Network
Cost in
Very Large Data
Centers
$13 / Mbps / month
Storage
$2.20 / GB / month
$0.40 / GB / month
5.7x
Administration
140 servers/admin
>1000 servers/admin
7.1x
Resource
Cost in
Medium Data Centers
27
Ratio
7.1x
Economics of Cloud
Providers (2)
Where
Possible Reasons
Why
3.6
Idaho
Hydroelectric power;
not sent long distance.
10.0
California
Electricity transmitted
long distance over the
grid;
limited transmission
lines in Bay Area; no
coal
fired electricity
allowed in California.
18.0
Hawaii
Economics of Cloud
Providers (3)
Extra benefits
Amazon: utilize off-peak capacity
Microsoft: sell .NET tools
Google: reuse existing infrastructure
Adoption Challenges
Challenge
Opportunity
Availability:
-Outages
-DDoS
Data lock-in
Standardization
30
Growth Challenges
Challenge
Data transfer bottlenecks
Performance unpredictability
Scalable storage
Bugs in large distributed systems
Scaling quickly
Opportunity
FedEx-ing disks, Data Backup/Archival
- Mailing disks is already provided by
Amazon
Improved VM support, flash memory,
scheduling VMs
Invent scalable store
Invent Debugger that relies on
Distributed VMs
Invent Auto-Scaler that relies on ML;
Snapshots
31
Opportunity
Offer reputation-guarding services like
those for email
Pay-for-use licenses; Bulk use sales
32