UNIT I Basic Concepts

Basic concepts-single layer perceptron-Multi layer perceptron-Adaline-Madaline- Learning

rulesSupervised learning-Back propagation networks-Training algorithm, Advanced
algorithms-Adaptive network- Radial basis network modular network-Applications

Basic concepts
● An artificial neuron network (neural network) is a computational model that mimics
the way nerve cells work in the human brain.
● Artificial neural networks (ANNs) use learning algorithms that can independently
make adjustments – or learn, in a sense – as they receive new input. This makes them
a very effective tool for non-linear statistical data modeling.
● Deep learning ANNs play an important role in machine learning (ML) and support the
broader field of artificial intelligence (AI) technology.

ANNs are composed of multiple nodes, which imitate biological neurons of human brain.
The neurons are connected by links and they
interact with each other. The nodes can take
input data and perform simple operations on
the data. The result of these operations is
passed to other neurons. The output at each
node is called its activation or node value.
Each link is associated with weight. ANNs are capable of learning, which takes place by
altering weight values. paper google sch

The architecture of an artificial neural network

To understand the concept of the architecture of an artificial neural network, we have to

understand what a neural network consists of. In order to define a neural network that
consists of a large number of artificial neurons, which are termed units arranged in a
sequence of layers. Lets us look at various types of layers available in an artificial neural
network.

Artificial Neural Network primarily consists of three layers:

Input Layer:

As the name suggests, it accepts inputs in

several different formats provided by the
programmer.

Hidden Layer:

The hidden layer presents in-between input and

output layers. It performs all the calculations to
find hidden features and patterns.

Output Layer:

The input goes through a series of transformations using the hidden layer, which finally
results in output that is conveyed using this layer.

The artificial neural network takes input and computes the weighted sum of the inputs and
includes a bias. This computation is represented in the form of a transfer function.
It determines weighted total is passed as an input to an activation function to produce the
output. Activation functions choose whether a node should fire or not. Only those who are
fired make it to the output layer. There are distinctive activation functions available that can
be applied upon the sort of task we are performing.

Types of Artificial Neural Networks

There are two Artificial Neural Network topologies − FeedForward and Feedback.

FeedForward ANN

In this ANN, the information flow is unidirectional. A

unit sends information to other unit from which it does
not receive any information. There are no feedback
loops. They are used in pattern
generation/recognition/classification. They have fixed
inputs and outputs.

FeedBack ANN

Here, feedback loops are allowed. They are used in content addressable memorie

Perceptrons

Frank Rosenblatt (1928 – 1971) was an American psychologist notable in the field of
Artificial Intelligence.

In 1957 he started something really big. He "invented" a Perceptron program, on an IBM 704
computer at Cornell Aeronautical Laboratory.
Scientists had discovered that brain cells (Neurons) receive input from our senses by
electrical signals.

The Neurons, then again, use electrical signals to store information, and to make decisions
based on previous input.

Frank had the idea that Perceptrons could simulate brain principles, with the ability to learn
and make decisions.

A perceptron is a neural network unit that does a precise

computation to detect features in the input data.
Perceptron is mainly used to classify the data into two
parts. Therefore, it is also known as Linear Binary
Classifier.

Perceptron uses the step function that returns +1 if the

weighted sum of its input 0 and -1.

The activation function is used to map the input between the required value like (0, 1) or (-1,
1).

A regular neural network looks like this:

The perceptron consists of 4 parts.

○ Input value or One input layer: The input layer of the perceptron is made of
artificial input neurons and takes the initial data into the system for further processing.

○ Weights and Bias:

Weight: It represents the dimension or strength of the connection between units. If the
weight to node 1 to node 2 has a higher quantity, then neuron 1 has a more
considerable influence on the neuron.
Bias: It is the same as the intercept added in a linear equation. It is an additional
parameter which task is to modify the output along with the weighted sum of the input
to the other neuron.

○ Net sum: It calculates the total sum.

○ Activation Function: A neuron can be activated or not, is determined by an

activation function. The activation function calculates a weighted sum and further
adding bias with it to give the result.

A standard neural network looks like the below diagram.

How does it work?

The perceptron works on these simple steps which are given below:

a. In the first step, all the inputs x are multiplied with their weights w.

b. In this step, add all the increased values and call them the Weighted sum.

c. In our last step, apply the weighted sum to a correct Activation Function.

For Example:

A Unit Step Activation Function

There are two types of architecture. These types focus on the functionality of artificial neural
networks as follows-

○ Single Layer Perceptron

○ Multi-Layer Perceptron

Single Layer Perceptron

It is one of the oldest and first introduced neural networks. It was proposed by Frank
Rosenblatt in 1958. Perceptron is also known as an artificial neural network. Perceptron is
mainly used to compute the logical gate like AND, OR, and NOR which has binary input
and binary output.

The main functionality of the perceptron is:-

● Takes input from the input layer

● Weight them up and sum it up.
● Pass the sum to the nonlinear function to produce the output.

Multi-layer Perceptron

Multi-Layer perceptron defines the most complex architecture of artificial neural networks. It
is substantially formed from multiple layers of the perceptron. TensorFlow is a very popular
deep learning framework released by, and this notebook will guide to build a neural network
with this library. If we want to understand what is a Multi-layer perceptron, we have to
develop a multi-layer perceptron from scratch using Numpy.
The pictorial representation of multi-layer perceptron learning is as shown below-

MLP networks are used for supervised learning format. A typical learning algorithm for MLP
networks is also called back propagation's algorithm.

A multilayer perceptron (MLP) is a feed forward artificial neural network that generates a set
of outputs from a set of inputs. An MLP is characterized by several layers of input nodes
connected as a directed graph between the input nodes connected as a directed graph between
the input and output layers. MLP uses backpropagation for training the network. MLP is a
deep learning method.

ADALINE

Adaline which stands for Adaptive Linear Neuron, is a network having a single linear unit. It
was developed by Widrow and Hoff in 1960. Some important points about Adaline are as
follows −

● It uses bipolar activation function.

● Adaline neuron can be trained using Delta rule or Least Mean Square(LMS) rule or
widrow-hoff rule
● The net input is compared with the target value to compute the error signal.
● on the basis of adaptive training algoritham weights are adjusted

The basic structure of Adaline is similar to perceptron having an extra feedback loop with the
help of which the actual output is compared with the desired/target output. After comparison
on the basis of training algorithm, the weights and bias will be updated.
Multiple Adaptive Linear Neuron (Madaline)

Madaline which stands for Multiple Adaptive Linear Neuron, is a network which consists of
many Adalines in parallel. It will have a single output unit. Some important points about
Madaline are as follows −

● It is just like a multilayer perceptron, where Adaline will act as a hidden unit between
the input and the Madaline layer.
● The weights and the bias between the input and Adaline layers, as in we see in the
Adaline architecture, are adjustable.
● The Adaline and Madaline layers have fixed weights and bias of 1.
● Training can be done with the help of Delta rule.

It consists of “n” units of input layer and “m” units of Adaline layer and “1” unit of the
Madaline layer. Each neuron in the Adaline and Madaline layers has a bias of excitation “1”.
The Adaline layer is present between the input layer and the Madaline layer; the Adaline
layer is considered as the hidden layer.
Learning rules

Learning rule enhances the Artificial Neural Network’s performance by applying this rule
over the network. Thus learning rule updates the weights and bias levels of a network when
certain conditions are met in the training process. it is a crucial part of the development of the
Neural Network.

Types Of Learning Rules in ANN

Let us see different learning rules in the Neural network:

● Hebbian learning rule – It identifies, how to modify the weights of nodes of a

network.
● Perceptron learning rule – Network starts its learning by assigning a random value to
each weight.
● Delta learning rule – Modification in sympatric weight of a node is equal to the
multiplication of error and the input.
● Correlation learning rule – The correlation rule is the supervised learning.
● Outstar learning rule – We can use it when it assumes that nodes or neurons in a
network arranged in a layer.

Supervised Machine Learning

Supervised learning is the types of machine learning in which machines are trained using well
"labelled" training data, and on basis of that data, machines predict the output. The labelled
data means some input data is already tagged with the correct output.
In supervised learning, the training data provided to the machines work as the supervisor that
teaches the machines to predict the output correctly. It applies the same concept as a student
learns in the supervision of the teacher.

Supervised learning is a process of providing input data as well as correct output data to the
machine learning model. The aim of a supervised learning algorithm is to find a mapping
function to map the input variable(x) with the output variable(y).

In the real-world, supervised learning can be used for Risk Assessment, Image
classification, Fraud Detection, spam filtering, etc.

How Supervised Learning Works?

In supervised learning, models are trained using labelled dataset, where the model learns
about each type of data. Once the training process is completed, the model is tested on the
basis of test data (a subset of the training set), and then it predicts the output.

The working of Supervised learning can be easily understood by the below example and
diagram:

Suppose we have a dataset of different types of shapes which includes square, rectangle,
triangle, and Polygon. Now the first step is that we need to train the model for each shape.

○ If the given shape has four sides, and all the sides are equal, then it will be labelled as
a Square.

○ If the given shape has three sides, then it will be labelled as a triangle.

○ If the given shape has six equal sides then it will be labelled as hexagon.
Now, after training, we test our model using the test set, and the task of the model is to
identify the shape.

The machine is already trained on all types of shapes, and when it finds a new shape, it
classifies the shape on the bases of a number of sides, and predicts the output.

Steps Involved in Supervised Learning:

○ First Determine the type of training dataset

○ Collect/Gather the labelled training data.

○ Split the training dataset into training dataset, test dataset, and validation dataset.

○ Determine the input features of the training dataset, which should have enough
knowledge so that the model can accurately predict the output.

○ Determine the suitable algorithm for the model, such as support vector machine,
decision tree, etc.

○ Execute the algorithm on the training dataset. Sometimes we need validation sets as
the control parameters, which are the subset of training datasets.

○ Evaluate the accuracy of the model by providing the test set. If the model predicts the
correct output, which means our model is accurate.

Types of supervised Machine learning Algorithms:

Supervised learning can be further divided into two types of problems:

1. Regression

Regression algorithms are used if there is a relationship between the input variable and the
output variable. It is used for the prediction of continuous variables, such as Weather
forecasting, Market Trends, etc. Below are some popular Regression algorithms which come
under supervised learning:

○ Linear Regression

○ Regression Trees

○ Non-Linear Regression

○ Bayesian Linear Regression

○ Polynomial Regression

2. Classification

Classification algorithms are used when the output variable is categorical, which means there
are two classes such as Yes-No, Male-Female, True-false, etc.

Spam Filtering,

○ Random Forest

○ Decision Trees

○ Logistic Regression

○ Support vector Machines

Advantages of Supervised learning:

○ With the help of supervised learning, the model can predict the output on the basis of
prior experiences.

○ In supervised learning, we can have an exact idea about the classes of objects.

○ Supervised learning model helps us to solve various real-world problems such as

fraud detection, spam filtering, etc.

Disadvantages of supervised learning:

 ○ Supervised learning models are not suitable for handling the complex tasks.

○ Supervised learning cannot predict the correct output if the test data is different from
the training dataset.

○ Training required lots of computation times.

○ In supervised learning, we need enough knowledge about the classes of object.

Back Propagation Neural Networks

Back Propagation Neural (BPN) is a multilayer neural network consisting of the input layer,
at least one hidden layer and output layer. As its name suggests, back propagating will take
place in this network. The error which is calculated at the output layer, by comparing the
target output and the actual output, will be propagated back towards the input layer.

Architecture

As shown in the diagram, the architecture of BPN has three interconnected layers having
weights on them. The hidden layer as well as the output layer also has bias, whose weight is
always 1, on them. As is clear from the diagram, the working of BPN is in two phases. One
phase sends the signal from the input layer to the output layer, and the other phase back
propagates the error from the output layer to the input layer.
Radial Basis Function Networks (RBFNs)

RBFNs are specific types of neural networks that follow a feed-forward approach and make
use of radial functions as activation functions. They consist of three layers namely the input
layer, hidden layer, and output layer which are mostly used for time-series prediction,
regression testing, and classification.

RBFNs do these tasks by measuring the similarities present in the training data set. They
usually have an input vector that feeds these data into the input layer thereby confirming the
identification and rolling out results by comparing previous data sets. Precisely, the input
layer has neurons that are sensitive to these data and the nodes in the layer are efficient in
classifying the class of data. Neurons are originally present in the hidden layer though they
work in close integration with the input layer. The hidden layer contains Gaussian transfer
functions that are inversely proportional to the distance of the output from the neuron's center.
The output layer has linear combinations of the radial-based data where the Gaussian
functions are passed in the neuron as parameter and output is generated. Consiider the given
image below to understand the process thoroughly.

Modular Neural Network

Applications of Modular Neural Network

1. Stock market prediction systems

2. Adaptive MNN for character recognitions
3. Compression of high level input data
A modular neural network has a number of different networks that function independently
and perform sub-tasks. The different networks do not really interact with or signal each other
during the computation process. They work independently towards achieving the output.

As a result, a large and complex computational process are done significantly faster by
breaking it down into independent components. The computation speed increases because the
networks are not interacting with or even connected to each other.

Applications of Neural Networks

Neural Networks are regulating some key sectors including finance, healthcare, and
automotive. As these artificial neurons function in a way similar to the human brain. They
can be used for image recognition, character recognition and stock market predictions. Let’s
understand the diverse applications of neural networks

1. Facial Recognition
Facial Recognition Systems are serving as robust systems of surveillance. Recognition
Systems matches the human face and compares it with the digital images. They are used in
offices for selective entries. The systems thus authenticate a human face and match it up with
the list of IDs that are present in its database.
2. Stock Market Prediction
Investments are subject to market risks. It is nearly impossible to predict the upcoming
changes in the highly volatile stock market. The forever changing bullish and bearish phases
were unpredictable before the advent of neural networks. But well what changed it all?
Neural Networks of course…
To make a successful stock prediction in real time a Multilayer Perceptron MLP (class of
feedforward artificial intelligence algorithm) is employed. MLP comprises multiple layers of
nodes, each of these layers is fully connected to the succeeding nodes. Stock’s past
performances, annual returns, and non profit ratios are considered for building the MLP
model

3. Social Media
No matter how cliche it may sound, social media has altered the normal boring course of life.
Artificial Neural Networks are used to study the behaviours of social media users. Data
shared everyday via virtual conversations is tacked up and analyzed for competitive analysis.
Neural networks duplicate the behaviours of social media users. Post analysis of individuals'
behaviours via social media networks the data can be linked to people’s spending habits.
Multilayer Perceptron ANN is used to mine data from social media applications

4. Aerospace
Aerospace Engineering is an expansive term that covers developments in spacecraft and
aircraft. Fault diagnosis, high performance auto piloting, securing the aircraft control
systems, and modeling key dynamic simulations are some of the key areas that neural
networks have taken over. Time delay Neural networks can be employed for modelling non
linear time dynamic systems.

5. Defence
Defence is the backbone of every country. Every country’s state in the international domain is
assessed by its military operations. Neural Networks also shape the defence operations of
technologically advanced countries. The United States of America, Britain, and Japan are
some countries that use artificial neural networks for developing an active defence strategy
.
6. Healthcare
The age old saying goes like “Health is Wealth”. Modern day individuals are leveraging the
advantages of technology in the healthcare sector. Convolutional Neural Networks are
actively employed in the healthcare industry for X ray detection, CT Scan and ultrasound.
7. Signature Verification and Handwriting Analysis
Signature Verification , as the self explanatory term goes, is used for verifying an
individual’s signature. Banks, and other financial institutions use signature verification to
cross check the identity of an individual.
8. Weather Forecasting

The forecasts done by the meteorological department were never accurate before artificial
intelligence came into force. Weather Forecasting is primarily undertaken to anticipate the
upcoming weather conditions beforehand. In the modern era, weather forecasts are even used
to predict the possibilities of natural disasters.