Artificial Neural Network Concepts/Terminology

ARTIFICIAL NEURAL NETWORK
CONCEPTS/TERMINOLOGY
• Inputs: Source data fed into the neural network, with the goal of making a decision or prediction about
the data. Inputs to a neural network are typically a set of real values; each value is fed into one of the
neurons in the input layer.
• Training Set: A set of inputs for which the correct outputs are known, used to train the neural network.
• Outputs : Neural networks generate their predictions in the form of a set of real values or boolean
decisions. Each output value is generated by one of the neurons in the output layer.
• Neuron/perceptron: The basic unit of the neural network. Accepts an input and generates a
prediction. Each neuron accepts part of the input and passes it through the activation function.
Common activation functions are sigmoid, TanH and ReLu. Activation functions help generate output
values within an acceptable range, and their non-linear form is crucial for training the network.
• Weight Space: Each neuron is given a numeric weight. The weights, together with the activation
function, define each neuron’s output. Neural networks are trained by fine-tuning weights, to discover
the optimal set of weights that generates the most accurate prediction.
• Forward Pass: The forward pass takes the inputs, passes them through the network and allows each
neuron to react to a fraction of the input. Neurons generate their outputs and pass them on to the
next layer, until eventually the network generates an output.
• Error Function: Defines how far the actual output of the current model is from the correct output.
When training the model, the objective is to minimize the error function and bring output as close as
possible to the correct value.
• Backpropagation: In order to discover the optimal weights for the neurons, we perform a
backward pass, moving back from the network’s prediction to the neurons that generated that
prediction. This is called backpropagation. Backpropagation tracks the derivatives of the
activation functions in each successive neuron, to find weights that bring the loss function to a
minimum, which will generate the best prediction. This is a mathematical process called
gradient descent.
• Bias and Variance: When training neural networks, like in other machine learning techniques,
we try to balance between bias and variance. Bias measures how well the model fits the
training set—able to correctly predict the known outputs of the training examples. Variance
measures how well the model works with unknown inputs that were not available during
training. Another meaning of bias is a “bias neuron” which is used in every layer of the neural
network. The bias neuron holds the number 1, and makes it possible to move the activation
function up, down, left and right on the number graph.
• Hyperparameters: A hyper parameter is a setting that affects the structure or operation of the
neural network. In real deep learning projects, tuning hyper parameters is the primary way to
build a network that provides accurate predictions for a certain problem. Common hyper
parameters include the number of hidden layers, the activation function, and how many times
(epochs) training should be repeated.
MCCULLOGH-PITTS MODEL
• In 1943 two electrical engineers, Warren McCullogh and Walter Pitts, published the first paper describing
what we would call a neural network
• It may be divided into 2 parts. The first part, g takes an input, performs an aggregation and based on the
aggregated value the second part, f makes a decision. Let us suppose that I want to predict my own
decision, whether to watch a random football game or not on TV. The inputs are all boolean i.e., {0,1} and
my output variable is also boolean {0: Will watch it, 1: Won’t watch it}.
• So, x1 could be ‘is Indian Premier League On’ (I like Premier League more)
• x2 could be ‘is it a knockout game (I tend to care less about the league level matches)
• x3 could be ‘is Not Home’(Can’t watch it when I’m in College. Can I?)
• x4 could be ‘is my favorite team playing’ and so on.
• These inputs can either be excitatory or inhibitory. Inhibitory inputs are those that have maximum effect on
the decision making irrespective of other inputs i.e., if x3 is 1 (not home) then my output will always be 0 i.e.,
the neuron will never fire, so x3 is an inhibitory input. Excitatory inputs are NOT the ones that will make the
neuron fire on their own but they might fire it when combined together. Formally, this is what is going on:
• We can see that g(x) is just doing a sum of the inputs — a simple aggregation. And  here is called thresholding
parameter. For example, if I always watch the game when the sum turns out to be 2 or more, the  is 2 here.
This is called the Thresholding Logic.
Implementing boolean functions
Circle at the end indicates inhibitory input: If any inhibitory input is 1 the output will be 0
• W0 is called the bias as it represents the prior
• A movie buff may have a low threshold and may watch any movie irrespective of the genre,
actor, director[=0]
• Selective viewer may only watch fantasies staring Prabhas and directed by Rajamouli [=3]
• The weights (w1,w2….wn) and the bias(w0) will depend on the data(viewer history)

Artificial Neural Network Concepts/Terminology

Uploaded by

Copyright:

Available Formats

Artificial Neural Network Concepts/Terminology

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Artificial Neural Network Concepts/Terminology

Uploaded by

Copyright:

Available Formats

ARTIFICIAL NEURAL NETWORK

You might also like