Cloud computing unit2 Rita Vadhvana

Unit -2
Erase of computing

The two fundamental and dominant model of computing are: Sequential and
parallel. The sequential computing are begun in 1940s; parallel (and distributed)
computing are followed it within a decade. The four key elements of computing
developed during these eras were: architectures, compilers, applications, and
problem – solving environments.

What is computing?

Computing is any goal-oriented activity requiring, benefiting from, or creating

computers. Computing includes designing, developing and building hardware
and software systems; designing a mathematical sequence of steps known as
an algorithm; processing, structuring and managing various kinds of
information.

Type of computing

1. Parallel Computing
In parallel computing multiple processors performs multiple tasks
assigned to them simultaneously. Memory in parallel systems can either
be shared or distributed. Parallel computing provides concurrency and
saves time and money.

2. Distributed Computing:
In distributed computing we have multiple autonomous computers which
seems to the user as single system. In distributed systems there is no
shared memory and computers communicate with each other through
message passing. In distributed computing a single task is divided among
different computers.

Difference between Parallel Computing and Distributed Computing:

S.NO Parallel Computing Distributed Computing

1. Many operations are performed simultaneously System components are
located at different locations
2. Single computer is required Uses multiple computers
Multiple computers perform
3. Multiple processors perform multiple operations multiple operations
4. It may have shared or distributed memory It have only distributed
Cloud computing unit2 Rita Vadhvana

memory

Computer communicates
with each other through
5. Processors communicate with each other through bus message passing.
Improves system scalability,
fault tolerance and resource
6. Improves the system performance sharing

Parallel Processing
Parallel processing can be described as a class of techniques which enables the system to achieve
simultaneous data-processing tasks to increase the computational speed of a computer system.

A parallel processing system can carry out simultaneous data-processing to achieve faster
execution time. For instance, while an instruction is being processed in the ALU component of
the CPU, the next instruction can be read from memory.

The primary purpose of parallel processing is to enhance the computer processing capability and
increase its throughput, i.e. the amount of processing that can be accomplished during a given
interval of time.

“A parallel processing system can be achieved by having a multiplicity of

functional units that perform identical or different operations simultaneously. The
data can be distributed among various multiple functional units.”

The following diagram shows one possible way of separating the execution unit into
eight functional units operating in parallel.

The operation performed in each functional unit is indicated in each block if the
diagram:
Cloud computing unit2 Rita Vadhvana

o The adder and integer multiplier performs the arithmetic operation with integer
numbers.
o The floating-point operations are separated into three circuits operating in
parallel.
o The logic, shift, and increment operations can be performed concurrently on
different data. All units are independent of each other, so one number can be
shifted while another number is being incremented.
Cloud computing unit2 Rita Vadhvana

Hardware architecture of parallel computing –

The hardware architecture of parallel computing is disturbed along the following
categories as given below:

1. Single-instruction, single-data (SISD) systems

2. Single-instruction, multiple-data (SIMD) systems
3. Multiple-instruction, single-data (MISD) systems
4. Multiple-instruction, multiple-data (MIMD) systems
Hardwar computing –
Computer hardware is the collection of physical parts of a computer system. This includes the computer
case, monitor, keyboard, and mouse. It also includes all the parts inside the computer case, such as the
hard disk drive, motherboard, video card, and many others. Computer hardware is what you can
physically touch.

1. Single-instruction, single-data (SISD) systems –

An SISD computing system is a uniprocessor machine which is capable of
executing a single instruction, operating on a single data stream. In SISD,
machine instructions are processed in a sequential manner and computers
adopting this model are popularly called sequential computers. Most
conventional computers have SISD architecture. All the instructions and data
to be processed have to be stored in primary memory.

The speed of the processing element in the SISD model is

limited(dependent) by the rate at which the computer can transfer
information internally. Dominant representative SISD systems are IBM PC,
workstations.
Cloud computing unit2 Rita Vadhvana

2. Single-instruction, multiple-data (SIMD) systems –

An SIMD system is a multiprocessor machine capable of executing the same
instruction on all the CPUs but operating on different data streams.
Machines based on an SIMD model are well suited to scientific computing
since they involve lots of vector and matrix operations. So that the
information can be passed to all the processing elements (PEs) organized
data elements of vectors can be divided into multiple sets(N-sets for N PE
systems) and each PE can process one data set.

Dominant representative SIMD systems is Cray’s vector processing

machine.
3. Multiple-instruction, single-data (MISD) systems –
An MISD computing system is a multiprocessor machine capable of
executing different instructions on different PEs but all of them operating on
the same dataset .
Cloud computing unit2 Rita Vadhvana

Example Z = sin(x)+cos(x)+tan(x)
The system performs different operations on the same data set. Machines
built using the MISD model are not useful in most of the application, a few
machines are built, but none of them are available commercially.
4. Multiple-instruction, multiple-data (MIMD) systems –
An MIMD system is a multiprocessor machine which is capable of executing
multiple instructions on multiple data sets. Each PE in the MIMD model has
separate instruction and data streams; therefore machines built using this
model are capable to any kind of application. Unlike SIMD and MISD
machines, PEs in MIMD machines work asynchronously.
Cloud computing unit2 Rita Vadhvana

MIMD machines are broadly categorized into shared-memory

MIMD and distributed-memory MIMD based on the way PEs are coupled to
the main memory.
a) shared memory MIMD Machine

In this model (tightly coupled multiprocessor systems), all the PEs are
connected to a single global memory and they all have access to it. The
communication between PEs in this model takes place through the
shared memory; modification of the data stored in the global memory by
one PE is visible to all other PEs. Dominant representative shared
memory MIMD systems are Silicon Graphics machines and Sun/IBM’s
SMP (Symmetric Multi-Processing).

b) Distributed memory MIMD Machine

In Distributed memory MIMD machines (loosely coupled multiprocessor

systems) all PEs have a local memory. The communication between PEs
in this model takes place through the interconnection network (the inter
process communication channel, or IPC). The network connecting PEs
can be configured to tree, mesh or in accordance with the requirement.
The shared-memory MIMD architecture is easier to program but is less
tolerant to failures and harder to extend with respect to the distributed
memory MIMD model. Failures in a shared-memory MIMD affect the
entire system, whereas this is not the case of the distributed model, in
Cloud computing unit2 Rita Vadhvana

which each of the PEs can be easily isolated. Moreover, shared memory
MIMD architectures are less likely to scale because the addition of more
PEs leads to memory contention. This is a situation that does not happen
in the case of distributed memory, in which each PE has its own memory.
As a result of practical outcomes and user’s requirement, distributed
memory MIMD architecture is superior to the other existing models.

Approaches to parallel programming

A sequential program is one that runs on a single processor and has a

single line of control. To make many processors collectively work on a
single program, the program must be divided into smaller independent
chunks so that each processor can work on separate chunks of the
problem. The program decomposed in this way is a parallel program.

A wide variety of parallel programming approaches are available. The

most prominent among them are the following:

• Data parallelism
• Process parallelism
• Parmer-and-worker model

These three models are all suitable for task-level parallelism. In the case
of data parallelism, the divide-and-conquer technique is used to split data
into multiple sets, and each data set is processed on different PGs using
the same instruction. This approach is highly suitable to processing on
machines based on the SIMD model.

In the case of process parallelism, a given operation has multiple (but

distinct) activities that can be processed on multiple activities. Which can
be processed on multiple process parallelism, a given operation has
multiple activities, which can be processed on multiple processors. In the
case of the farmer-and-worker model, a job distribution approach is used:
one processor is configured as master and all other remaining PEs are
designated as slaves; the master assigns jobs to slave PGs and, on
Completion informs the master, which in tum collects results. These
approaches can be utilized in different levels of parallelism.
Cloud computing unit2 Rita Vadhvana

 Element of Distribute computing

1) General concepts and definition

Distributed computing studies the model, the architectures, and the

algorithms used for building and managing distributed systems.

“A distributed system is a collection of independent computers that

appears to its users as single coherent systems.”

“A distributed system is one in which components located at networked

computers communicate and coordinate their action only by passing
messages”.

2) Component of a Distributed System

A distributed system is the result of the interaction of several components

that traverse the entire computing stack from hardware to software.

It emerges from the collaboration of several elements that by working

together give to the users the illusion of a single coherent system.

At the very bottom layer, computer and network hardware constitute the
physical infrastructure; these components are directly managed by the
operating system that provides the basic services for: inter-process
communication, process scheduling and management and resource
management in terms of file system and local devices.
Cloud computing unit2 Rita Vadhvana

The use of well-known standards at the operating system, and even more at the
hardware and network level, allows easy harnessing of heterogeneous and their
organization into a coherent and uniform system.

The middleware layer leverages such services to build a uniform environment for
the development and deployment of distributed applications. This layer supports
the programming paradigms for distributed systems.

The top of the distributed system stack is represented by the applications and
services designed and developed to use the middleware. These can serve
several purposes and often expose their feature in the form of graphical user
interface accessible locally or through the internet via a web browser.
Cloud computing unit2 Rita Vadhvana

3) Architectural Styles for Distributed Computing

1. Component and connectors
 A component represents a unit of software that encapsulates a
function or a feature of the system.
 Example of components can be programs, objects, processes, pipes and
filters.
 A connector is a communication mechanism that allows the cooperation
and coordination among components.
2. Software Architecture Style
 Software architecture styles are based on the logical arrangement of
software component.
 They are helpful because they provide an intuitive view of the whole
system, despite its physical deployment.
 They also identify what are the main abstractions that are used to shape
the component of the system, and what are the expected interaction
patterns between them.

a) Data –centered Architecture

In data-centered architecture, the data is centralized and accessed frequently by other
components, which modify data. The main purpose of this style is to achieve integrality
of data. Data-centered architecture consists of different components that communicate
through shared data repositories. The components access a shared data structure and
are relatively independent, in that, they interact only through the data store.
The most well-known examples of the data-centered architecture is a database
architecture, in which the common database schema is created with data definition
protocol – for example, a set of related tables with fields and data types in an RDBMS.
Another example of data-centered architectures is the web architecture which has a
common data schema (i.e. meta-structure of the Web) and follows hypermedia data
model and processes communicate through the use of shared web-based data
services.
Cloud computing unit2 Rita Vadhvana

Types of Components
There are two types of components −
 A central data structure or data store or data repository, which is responsible for
providing permanent data storage. It represents the current state.
 A data accessory or a collection of independent components that operate on
the central data store, perform computations, and might put back the results.
Interactions or communication between the data accessory is only through the data
store. The data is the only means of communication among clients. The flow of control
differentiates the architecture into two categories −

 Repository Architecture Style

 Blackboard Architecture Style

Repository Architecture Style

In Repository Architecture Style, the data store is passive and the clients (software
components or agents) of the data store are active, which control the logic flow. The
participating components check the data-store for changes.
 The client sends a request to the system to perform actions (e.g. insert data).
 The computational processes are independent and triggered by incoming
requests.
 If the types of transactions in an input stream of transactions trigger selection of
processes to execute, then it is traditional database or repository architecture, or
passive repository.
 This approach is widely used in DBMS, library information system, the interface
repository in CORBA, compilers and CASE (computer aided software
engineering) environments.

Advantages
 Provides data integrity, backup and restore features.
 Provides scalability and reusability of agents as they do not have direct
communication with each other.
 Reduces overhead of transient data between software components.
Cloud computing unit2 Rita Vadhvana

Disadvantages
 It is more vulnerable to failure and data replication or duplication is possible.
 High dependency between data structure of data store and its agents.
 Changes in data structure highly affect the clients.
 Evolution of data is difficult and expensive.
 Cost of moving data on network for distributed data.

Blackboard Architecture Style

In Blackboard Architecture Style, the data store is active and its clients are passive.
Therefore the logical flow is determined by the current data status in data store. It has
a blackboard component, acting as a central data repository, and an internal
representation is built and acted upon by different computational elements.
 A number of components that act independently on the common data structure
are stored in the blackboard.
 In this style, the components interact only through the blackboard. The data-
store alerts the clients whenever there is a data-store change.
 The current state of the solution is stored in the blackboard and processing is
triggered by the state of the blackboard.
 The system sends notifications known as trigger and data to the clients when
changes occur in the data.
 This approach is found in certain AI applications and complex applications, such
as speech recognition, image recognition, security system, and business
resource management systems etc.
 If the current state of the central data structure is the main trigger of selecting
processes to execute, the repository can be a blackboard and this shared data
source is an active agent.
 A major difference with traditional database systems is that the invocation of
computational elements in a blackboard architecture is triggered by the current
state of the blackboard, and not by external inputs.

Parts of Blackboard Model

The blackboard model is usually presented with three major parts −
Knowledge Sources (KS)
Knowledge Sources, also known as Listeners or Subscribers are distinct and
independent units. They solve parts of a problem and aggregate partial results.
Interaction among knowledge sources takes place uniquely through the blackboard.
Blackboard Data Structure
Cloud computing unit2 Rita Vadhvana

The problem-solving state data is organized into an application-dependent hierarchy.

Knowledge sources make changes to the blackboard that lead incrementally to a
solution to the problem.
Control
Control manages tasks and checks the work state.

Advantages
 Provides scalability which provides easy to add or update knowledge source.
 Provides concurrency that allows all knowledge sources to work in parallel as
they are independent of each other.
 Supports experimentation for hypotheses.
 Supports reusability of knowledge source agents.

Disadvantages
 The structure change of blackboard may have a significant impact on all of its
agents as close dependency exists between blackboard and knowledge source.
 It can be difficult to decide when to terminate the reasoning as only approximate
solution is expected.
 Problems in synchronization of multiple agents.
 Major challenges in designing and testing of system.

b) Data – Flow Architecture

In data flow architecture, the whole software system is seen as a series of

transformations on consecutive pieces or set of input data, where data and operations
are independent of each other. In this approach, the data enters into the system and
then flows through the modules one at a time until they are assigned to some final
destination (output or a data store).
The connections between the components or modules may be implemented as I/O
stream, I/O buffers, piped, or other types of connections. The data can be flown in the
graph topology with cycles, in a linear structure without cycles, or in a tree type
structure.
The main objective of this approach is to achieve the qualities of reuse and
modifiability. It is suitable for applications that involve a well-defined series of
independent data transformations or computations on orderly defined input and output
Cloud computing unit2 Rita Vadhvana

such as compilers and business data processing applications. There are three types of
execution sequences between modules−

 Batch sequential
 Pipe and filter or non-sequential pipeline mode
 Process control

Batch Sequential
Batch sequential is a classical data processing model, in which a data transformation
subsystem can initiate its process only after its previous subsystem is completely
through −
 The flow of data carries a batch of data as a whole from one subsystem to
another.
 The communications between the modules are conducted through temporary
intermediate files which can be removed by successive subsystems.
 It is applicable for those applications where data is batched, and each
subsystem reads related input files and writes output files.
 Typical application of this architecture includes business data processing such
as banking and utility billing.

Advantages
 Provides simpler divisions on subsystems.
 Each subsystem can be an independent program working on input data and
producing output data.

Disadvantages
 Provides high latency and low throughput.
 Does not provide concurrency and interactive interface.
 External control is required for implementation.

Pipe and Filter Architecture

This approach lays emphasis on the incremental transformation of data by successive
component. In this approach, the flow of data is driven by data and the whole system is
decomposed into components of data source, filters, pipes, and data sinks.
The connections between modules are data stream which is first-in/first-out buffer that
can be stream of bytes, characters, or any other type of such kind. The main feature of
this architecture is its concurrent and incremented execution.
Cloud computing unit2 Rita Vadhvana

Filter
A filter is an independent data stream transformer or stream transducers. It transforms
the data of the input data stream, processes it, and writes the transformed data stream
over a pipe for the next filter to process. It works in an incremental mode, in which it
starts working as soon as data arrives through connected pipe. There are two types of
filters − active filter and passive filter.
Active filter
Active filter lets connected pipes to pull data in and push out the transformed data. It
operates with passive pipe, which provides read/write mechanisms for pulling and
pushing. This mode is used in UNIX pipe and filter mechanism.
Passive filter
Passive filter lets connected pipes to push data in and pull data out. It operates with
active pipe, which pulls data from a filter and pushes data into the next filter. It must
provide read/write mechanism.

Advantages
 Provides concurrency and high throughput for excessive data processing.
 Provides reusability and simplifies system maintenance.
 Provides modifiability and low coupling between filters.
 Provides simplicity by offering clear divisions between any two filters connected
by pipe.
 Provides flexibility by supporting both sequential and parallel execution.

Disadvantages
 Not suitable for dynamic interactions.
 A low common denominator is needed for transmission of data in ASCII formats.
 Overhead of data transformation between filters.
 Does not provide a way for filters to cooperatively interact to solve a problem.
 Difficult to configure this architecture dynamically.
Cloud computing unit2 Rita Vadhvana

Pipe
Pipes are stateless and they carry binary or character stream which exist between two
filters. It can move a data stream from one filter to another. Pipes use a little contextual
information and retain no state information between instantiations.

Process Control Architecture

It is a type of data flow architecture where data is neither batched sequential nor
pipelined stream. The flow of data comes from a set of variables, which controls the
execution of process. It decomposes the entire system into subsystems or modules
and connects them.

Types of Subsystems
Process control architecture would have a processing unit for changing the process
control variables and a controller unit for calculating the amount of changes.
A controller unit must have the following elements −
 Controlled Variable − Controlled Variable provides values for the underlying
system and should be measured by sensors. For example, speed in cruise
control system.
 Input Variable − Measures an input to the process. For example, temperature of
return air in temperature control system
 Manipulated Variable − Manipulated Variable value is adjusted or changed by
the controller.
 Process Definition − It includes mechanisms for manipulating some process
variables.
 Sensor − Obtains values of process variables pertinent to control and can be
used as a feedback reference to recalculate manipulated variables.
 Set Point − It is the desired value for a controlled variable.
 Control Algorithm − It is used for deciding how to manipulate process
variables.

Application Areas
Process control architecture is suitable in the following domains −
 Embedded system software design, where the system is manipulated by process
control variable data.
 Applications, which aim is to maintain specified properties of the outputs of the
process at given reference values.
 Applicable for car-cruise control and building temperature control systems.
Cloud computing unit2 Rita Vadhvana

 Real-time system software to control automobile anti-lock brakes, nuclear power

plants, etc.

5. virtual Machine Architecture

This class of architecture styles is characterized by the presence of an
abstract execution environment that simulates features that are not available
in the hardware or software.

Application and system are implemented on top of this layer and become
portable over different hardware and software environment as long as there
is implementation of virtual machine they interface with.

Rule – based style

These architectures characterized by representing the abstract execution

environment as an interface again.

Interpreter style