Introduction to Distributed

System

Chapter One: Introduction

 Outline
• Introduction and Definition
• Goals of a Distributed System
• Types of Distributed Systems
• Architectural Styles
• System Architectures
 1.1 Introduction
• Before the mid-80s, computers were
– Large and very expensive (hundred of thousands or even
millions of dollars)
– very slow (a few thousand instructions per second)
– not connected among themselves
• After the mid-80s: two major developments
– cheap and powerful microprocessor-based computers appeared
– high-speed computer networks
• LANs at speeds ranging from 10 to 1000 Mbps
• WANs at speed ranging from 64 Kbps to gigabits/sec
• Consequence
– feasibility of using a large network of computers to work for the
same application; this is in contrast to the old centralized
systems where there was a single computer with its peripherals
 Distributed System: Definition
A distributed system is a collection of independent computers that
appears to its users as a single coherent system.
(Tanenbaum & Van Steen)
• Two aspects:
(1) hardware - autonomous computers
(2) software – users think they are dealing with a single system
 Why Distributed?
• Resource and Data Sharing
– printers, databases, multimedia servers, ...
• Availability, Reliability
– the loss of some instances can be hidden
• Scalability, Extensibility
– the system grows with demand (e.g., extra servers)
• Performance
– huge power (CPU, memory, ...) available
• Inherent distribution, communication
– organizational distribution, e-mail, video
 Problems of Distribution
• Concurrency, Security
– clients must not disturb each other
• Privacy
– unwanted communication such as spam
• Partial failure
– we often do not know where the error is (e.g., RPC)
• Location, Migration, Replication
– clients must be able to find their servers
• Heterogeneity
– hardware, platforms, languages, management
 Characteristics of Distributed
Systems
• differences between the computers and the way
they communicate are hidden from users.
• users and applications can interact with a
distributed system in a consistent and uniform
way regardless of location.
• distributed systems should be easy to expand
and scale.
• a distributed system is normally continuously
available, even if there may be partial failures.
 1.2 Organization and Goals of a
Distributed System
• To support heterogeneous computers and networks and to
provide a single-system view, a distributed system is often
organized by means of a layer of software called middleware
that extends over multiple machines

a distributed system organized as middleware; note that the middleware

layer extends over multiple machines
 Goals of a distributed system
• Allow users to access and share resources(printers, computers,
storage facilities, data, files, Web pages, ...)
– reasons: economics, to collaborate and exchange information
• Transparency
– To hide the fact that its processes and resources are physically
distributed across multiple computers
– A distributed system that is able to present itself to users and
applications as if it were only a single computer system is said to
be transparent.
• Openness
– To offer services according to standard rules that describe the
syntax and semantics of those services
– E.g., specify interfaces using an interface definition language
(IDL)
• Scalability
• Different forms of transparency in a distributed system
Transparency Description
Access Hide differences in data representation
(endianness, file naming, ...) and how a resource
is accessed.
Location Hide where a resource is physically located; where
is http://www.prenhall.com/index.html? (naming)
Migration Hide that a resource may move to another location
Relocation Hide that a resource may be moved to another
location while in use; e.g., mobile users using their
wireless laptops
Replication Hide that a resource is replicated
Concurrency Hide that a resource may be shared by several
competitive users; a resource must be left in a
consistent state
Failure Hide the failure and recovery of a resource
Persistence Hide whether a (software) resource is in memory
or on disk
 Openness of Distributed
Systems
• Open distributed system: Be able to interact with services from
other open systems, irrespective of the underlying environment:
• Systems should conform to well-defined interfaces
• Systems should support portability of applications
• Systems should easily interoperate
• Achieving openness: At least make the distributed system
independent from heterogeneity of the underlying environment:
• Hardware
• Platforms
• Languages
 Policies versus Mechanisms
• Implementing openness: Requires support for different policies
specified by applications and users:
– What level of consistency do we require for client cached data?
– Which operations do we allow downloaded code to perform?
– Which QoS requirements do we adjust in the face of varying
bandwidth?
– What level of secrecy do we require for communication?
• Implementing openness: Ideally, a distributed system provides
only mechanisms:
– Allow (dynamic) setting of caching policies, preferably per
cachable item
– Support different levels of trust for mobile code
– Provide adjustable QoS parameters per data stream
– Offer different encryption algorithms
 Scalability in Distributed
Systems
• Observation: Many developers of modern distributed systems
easily use the adjective “scalable” without making clear why their
system actually scales.
• Scalability: At least three components:
• Number of users and/or processes (size scalability)
• Maximum distance between nodes (geographical scalability)
• Number of administrative domains (administrative scalability)
• Most systems account only, to a certain extent, for size scalability.
The (non)solution: powerful servers.
• Today, the challenge lies in geographical and administrative
scalability.
 Scalability Problems
Concept Example
Single server for all users-mostly for security
Centralized services
reasons
Centralized data A single on-line telephone book
Centralized algorithms Doing routing based on complete information

examples of scalability limitations

 Scaling Techniques
• how to solve scaling problems
• the problem is mainly performance, and arises
as a result of limitations in the capacity of
servers and networks (for geographical
scalability)
• three possible solutions: hiding
communication latencies, distribution, and
replication
 A.Hide Communication
Latencies
• try to avoid waiting for responses to remote service
requests
• let the requester do other useful job
• i.e., construct requesting applications that use only
asynchronous communication instead of synchronous
communication; when a reply arrives the application is
interrupted
• good for batch processing and parallel applications but
not for interactive applications
• for interactive applications, move part of the job to the
client to reduce communication; e.g. filling a form and
checking the entries
 (a) a server checking the correctness of field entries
(b) a client doing the job
 e.g., shipping code is now supported in Web applications using Java Applets
 B. Distribution
• e.g., DNS - Domain Name System
(ayalew.belay@cs.aau.edu.et)
• divide the name space into zones

an example of dividing the DNS name space into zones

 C. Replication
• replicate components across a distributed
system to increase availability and for load
balancing, leading to better performance
• decided by the owner of a resource
• caching (a special form of replication) also
reduces communication latency; decided by the
user
• but, caching and replication may lead to
consistency problems (see Chapter 6 -
Consistency and Replication)
 1.3 Types of distributed
Systems
• Three types:
– distributed computing systems,
– distributed information systems, and
– pervasive/embedded systems
1. Distributed Computing Systems
– Used for high-performance computing tasks
– two types: cluster computing and grid
computing
 Distributed Computing Systems
-Cluster Computing
• a collection of similar workstations or PCs
(homogeneous), closely connected by
means of a high-speed LAN
• each node runs the same operating
system
• used for parallel programming in which a
single compute intensive program is run in
parallel on multiple machines
 Distributed Computing Systems
-Cluster Computing

an example of a cluster computing system

 a master node runs a middleware (containing libraries
for parallel programs) and controls other compute
nodes;
 it allocates tasks
 provides an interface to users ,etc.
Distributed Computing Systems
• Grid Computing
– “Resource sharing and coordinated problem solving in dynamic,
multi-institutional virtual organizations" Ian Foster)
– high degree of heterogeneity: no assumptions are made
concerning hardware, operating systems, networks,
administrative domains, security policies, etc.
– Globus is a software sytem for Grid Computing; read about the
Globus Alliance at http://www.globus.org/
• Cloud Computing
– the use of computing resources (hardware and software) that are
delivered as a service over a network (typically the Internet)
 2.Distributed Information Systems

• problem: many networked applications with a

problem of interoperability
• at the lowest level: wrap a number of requests
into a single larger request and have it executed
as a distributed transaction; all or none of the
requests would be executed
• how to let applications communicate directly with
each other, i.e., Enterprise Application
Integration (EAI)
 Transaction Processing Systems
• applications
• special primitives are required Transaction Processing Systems
• consider database to program transactions, supplied either by the
underlying distributed system or by the language runtime system
• exact list of primitives depends on the type of application; procedure
calls, ordinary statements, etc. can also be included
• e.g., assume the following banking operation
– withdraw an amount x from account 1
– deposit the amount x to account 2
• what happens if there is a problem after the first activity is carried
out?
• group the two operations into one transaction; either both are carried
out or neither
• we need a way to roll back when a transaction is not completed
 Transaction Processing Systems
• Properties of transactions, often referred to as ACID
1. Atomic: to the outside world, the transaction happens indivisibly; a
transaction either happens completely or not at all; intermediate
states are not seen by other processes
2. Consistent: the transaction does not violate system invariants;
e.g., in an internal transfer in a bank, the amount of money in the
bank must be the same as it was before the transfer (the law of
conservation of money); this may be violated for a brief period of
time, but not seen to other processes
3. Isolated or Serializable: concurrent transactions do not interfere
with each other; if two or more transactions are running at the same
time, the final result must look as though all transactions run
sequentially in some order
4. Durable: once a transaction commits, the changes are permanent;
see later in Chapter 8 -Fault Tolerance
 3.Distributed Pervasive Systems
• the distributed systems discussed so far are characterized by their
stability; fixed nodes having high-quality connection to a network
• there are also mobile and embedded computing devices which are
small, battery-powered, mobile, and with a wireless connection
• three requirements for pervasive applications
– embrace contextual changes: a device is aware that its environment
may change all the time, e.g., changing its network access point
– encourage ad hoc composition: devices are used in different ways by
different users
– recognize sharing as the default: devices join a system to access or
provide information
• examples of pervasive systems
– Home Systems that integrate consumer electronics
– Electronic Health Care Systems to monitor the well-being of individuals
– Sensor Networks
– read pages 26 - 30
 Distributed Systems Examples
(The Internet)
• The Internet is a vast interconnected collection of
computer networks of many types.
• Its design enabling a program running anywhere to
address messages to programs anywhere else.
• Allowing its users to make use of many services as:
WWW, E-Mail, Web hosting, and File transfer.
• Its services can be extended by adding new types of
service (open-ended services).
• Small organizations and individual users can to access
internet services through Internet Service Providers
(ISPs).
• Independent intranets are linked together by high
transmission capacity circuits called backbones.
 Distributed Systems Examples
(The Internet)

 intranet  %
 %
 %  ISP

 %

 backbone

 satellite link

 desktop computer:
server:
network link:

A typical portion of the Internet

 Distributed Systems Examples
(Intranets)
• An Intranet is a portion of the internet that is
administrated separately and its local security policies
are enforced by a configured boundary.
• Composed of several local area networks (LANs) linked
by backbone connections to allow its users to access the
provided services.
• Connected to the Internet via a router which allows its
users to make use of the internet services elsewhere.
• Many organization protect their own services from
unauthorized use by filtering incoming and outgoing
messages using a firewall.
 Distributed Systems Examples
(Intranets)
email serv er Desk top
com put ers
print and other serv ers

Loc al area
Web serv er network

email serv er
print
File s erv er
other s erv ers
the res t of
the Internet
router/firewall

A typical Intranet
 Distributed Systems Examples
(Mobile and Ubiquitous Computing)
• The portability of many computing devices and the
ability to connect to networks in different places
makes mobile computing possible.
• Mobile computing is the performance of computing
tasks while the users are on the move and away from
their residence intranet but still provided with access
to resources via the devices they carry with them.
• Ubiquitous computing is the harnessing of many
small cheap computational devices that are present
in user’s physical environments.
• Ubiquitous and mobile computing overlap but they
are generally distinct.
 Distributed Systems Examples
(Mobile and Ubiquitous Computing)
Internet

Host intranet WAP

Wireless LAN Home intranet
gateway

Mobile
phone
Printer Laptop
Camera Host site

Portable and handheld devices in a distributed system

 1.4 Architectural Styles
• To master the complexity of distributed systems, it is crucial that they
are properly organized
• Concern logical organization of distributed systems into software
components and connectors
– Components are replaceable units within its environment
– Connectors are mechanisms that mediate communication,
coordination and cooperation among components e.g., facilities
for RPC, message passing, or streaming multimedia data
• Important architectural styles for DS
– Layered architectures
– Object-based architectures
– Data-centered architectures
– Event-based architectures
 Layered architecture

• components are organized in a layered fashion where a

component at layer Li is allowed to call components at
the underlying layer Li-1, but not the other way around;
• requests go down the hierarchy and results flow upward
• e.g., network layers

the layered architectural style

 Object-based architecture

• each object corresponds to a component and these

components are connected through a remote procedure
call mechanism (matches the client-server paradigm)

the object-based architectural style

 Event-based architecture

• processes communicate through the propagation of

events (can also optionally carry data)
• publish/subscribe systems
• processes publish events and the middleware ensures
that only those processes that subscribed to those
events will receive them
• processes are loosely
coupled; no need of
explicitly referring to
each other.
 A data-centered architecture

• processes communicate through a common repository;

e.g., a shared distributed file system
• Shared data-spaces:
– event-based architectures combined with data-centered
architectures
– processes are decoupled in time

the shared data-space architectural style

 1.5 System Architectures

• refers to the logical organization of distributed

systems into software components or how are
processes organized in a system; where do we
place software components
• deciding on software components, their
interaction, and their placement is what system
architecture is all about
• can be centralized, decentralized or a hybrid
 System Architectures
• Centralized: traditional client-server structure
– Vertical (or hierarchical) organization of communication and
control paths (as in layered software architectures)
– Logical separation of functions into client (requesting process)
and server (responder)
• Decentralized: peer-to-peer
– Horizontal rather than hierarchical comm. and control
• Hybrid: combine elements of C/S and P2P
– Edge-server systems
– Collaborative distributed systems.
• Classification of a system as centralized or decentralized
refers to communication and control organization,
primarily.
 Centralized Architectures

• thinking in terms of clients requesting services from

servers
• a server is a process implementing a specific service
• a client is a process that requests a service from a
server by sending a request and waiting for a reply
• we have a request-reply behavior

Server and client Interaction

 communication between client and server can be
◦ by a connectionless protocol if the underlying network is fairly
reliable; efficient since there is no much overhead
◦ but assuring reliability is difficult
◦ we don’t also know the source of error; was the request or the reply
lost, for instance
◦ when messages are lost or corrupted let the client send the request
again; applicable only for idempotent operations
◦ an operation is idempotent if it can be repeated multiple times
without harm; e.g., reading a record in a database
◦ but, transferring an amount to a bank account is not idempotent
◦ see later in Chapter 8 -Fault Tolerance
◦ by a reliable connection-oriented protocol if the underlying
◦ network is unreliable
◦ establishing and terminating connections is expensive
 Client-Server Architectures

• Processes are divided into two groups (clients

and servers).
• Synchronous communication: request-reply
protocol
• In LANs, often implemented with a
connectionless protocol (unreliable)
• In WANs, communication is typically connection-
oriented TCP/IP (reliable)
– High likelihood of communication failures
 Application Layering

• User-interface level: GUI’s (usually) for

interacting with end users
• Processing level: data processing applications
– the core functionality
• Data level: interacts with data base or file
system
– Data usually is persistent; exists even if no client is
accessing it
– File or database system
 Examples
• Web search engine
– Interface: type in a keyword string
– Processing level: processes to generate DB queries, rank replies,
format response
– Data level: database of web pages
• Stock broker’s decision support system
– Interface: likely more complex than simple search
– Processing: programs to analyze data; rely on statistics, AI perhaps,
may require large simulations
– Data level: DB of financial information
• Desktop “office suites”
– Interface: access to various documents, data,
– Processing: word processing, database queries, spreadsheets,…
– Data : file systems and/or databases
Application Layering
 System Architecture

• Mapping the software architecture to system

hardware
– Correspondence between logical software modules
and actual computers
• Multi-tiered architectures
– Layer and tier are roughly equivalent terms, but layer
typically implies software and tier is more likely to
refer to hardware.
– Two-tier and three-tier are the most common
 Two-tiered C/S Architectures

• Server provides processing and data

management; client provides simple graphical
display (thin-client)
– Perceived performance loss at client
– Easier to manage, more reliable, client machines
don’t need to be so large and powerful
• At the other extreme, all application processing
and some data resides at the client (fat-client
approach)
– Pro: reduces work load at server; more scalable
– Con: harder to manage by system admin, less secure
 Two-tiered C/S Architectures

• how to physically distribute a client-server

application across several machines

Two-tiered architecture: alternative client-server organizations

 Two-tiered C/S Architectures

(a) put only terminal-dependent part of the user interface on

the client machine and let the applications remotely
control the presentation
(b) put the entire user-interface software on the client side
(c) move part of the application to the client, e.g. checking
correctness in filling forms
(a) to (c) are for thin clients
(d) and (e) are for powerful client machines what are called
fat clients (more popular)
(d) and (e) are difficult to manage since client side software
is distributed and is prone to error; it is also dependent
on the client’s platform such as operating system
 Three-tiered Architectures

• In some applications servers may also need to

be clients, leading to a three level architecture
– Distributed transaction processing
– Web servers that interact with database servers
• Distribute functionality across three levels of
machines instead of two.
 Decentralized Architectures

• vertical distribution: refers to the ones discussed so far

where the different tiers correspond directly with the
logical organization of applications; place logically
different components on different machines
• horizontal distribution: physically split up the client or
the server into logically equivalent parts
• an example is a peer-to-peer system where processes
are equal and hence each process acts as a client and a
server at the same time (servant)
• read about the different approaches of peer-to-peer
architecture and about Architectures versus Middleware
 Decentralized Architectures

• another example is the horizontal distribution of

a Web service
 Centralized v Decentralized
Architectures
• Traditional client-server architectures exhibit
vertical distribution. Each level serves a
different purpose in the system.
– Logically different components reside on different
nodes
• Horizontal distribution (P2P): each node has
roughly the same processing capabilities and
stores/manages part of the total system data.
– Better load balancing, more resistant to denial-of-
service attacks, harder to manage than C/S
– Communication & control is not hierarchical; all about
equal
Thank you!