Nothing Special   »   [go: up one dir, main page]
Scribd Logo
Upload

Welcome to Scribd!

  • 17 views

    Day 10

    Uploaded by

    Sulakx Kuru
    The document provides an overview of different deep learning algorithms including artificial neural networks, recurrent neural networks, and convolutional neural networks. It describes their basic architectures and applications. Implementation details like softmax, backpropagation, and batch training are also covered.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd

    Day 10

    Uploaded by

    Sulakx Kuru
    17 views17 pages
    The document provides an overview of different deep learning algorithms including artificial neural networks, recurrent neural networks, and convolutional neural networks. It describes their basic architectures and applications. Implementation details like softmax, backpropagation, and batch training are also covered.

    Original Title

    Day_10

    Copyright

    © © All Rights Reserved

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document provides an overview of different deep learning algorithms including artificial neural networks, recurrent neural networks, and convolutional neural networks. It describes their basic architectures and applications. Implementation details like softmax, backpropagation, and batch training are also covered.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    17 views17 pages

    Day 10

    Uploaded by

    Sulakx Kuru
    The document provides an overview of different deep learning algorithms including artificial neural networks, recurrent neural networks, and convolutional neural networks. It describes their basic architectures and applications. Implementation details like softmax, backpropagation, and batch training are also covered.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    You are on page 1of 17

    Prerequisites for this

    Course
    Laptop with Good Internet
    No basic knowledge is required.
    Note & Pen
    Deep Learning Algorithm
    ARTIFICIAL NEURAL
    01
    NETWORK (ANN)

    RECURRENT NEURAL
    02
    NETWORK (RNN)

    CONVOLUTIONAL
    03
    NEURAL NETWORK (CNN)
    ANN
    • Learns any Non-Linear Function, It is known as
    Universal Function Approximators Hidden
    Input
    • Activation Function introduce non linear
    property to network, so it will identify complex
    relationship between input & output Output

    • Output of each neuron is the activation of


    weighted sum of Input, If there is no Activation
    function, network can't learn non-linear function

    • Feed Forward Neural Network – Input processed


    in one direction

    • When hidden layer is more than one, that is Deep


    Neural Network
    RNN
    Hidden
    • Looping system in hidden layer of ANN is Input
    known as RNN

    • It captures sequential info of input data, that is Output


    dependency between words to make
    prediction. Whereas, ANN cannot capture
    sequential information

    • It is mostly used in NLP (Natural Language


    Processing)
    CNN
    • CNN learns the filter automatically to extract the right features from the data

    • It captures spatial features (Arrangement of pixels) whereas ANN can’t.

    • It don’t have recurrent connections like RNN, instead it has convolution type of
    hidden layers

    • Convolution and pooling functions are used as activation functions

    • CONVOLUTION: Input image and other as Filter on input image(Kernel)


    produces output image.

    • POOLING: picking maximum value from selected region is Max pooling and vice
    versa.
    CNN Architecture FC_4
    FC_3 Fully Connected
    Fully Connected Neural Network
    Max-pooling
    0
    Conv_2
    Convolution (2x2) ReLU Activation
    (5x5)
    Conv_1
    Convolution
    (5x5)
    Max-pooling
    (2x2) 1
    .
    .
    2
    .
    . 9
    Output
    Input
    28x28x1 n1 channels n1 channels n2 channels Flattened
    n3 units
    (24 x 24 x n1) (12 x 12 x n1) (8 x 8 x n2) n2 channels
    (4 x 4 x n2)
    Simple Softmax Classification
    Simple Softmax Classification
    784 Pixels

    ….

    .
    .
    .
    .
    Input
    28x28x1

    .
    .
    .
    .
    0 1 2 3 9
    100 image at a time
    In TensorFlow
    SOFTMAX Function

    • Softmax activation function will be applied in the last layer of


    Neural network, instead of ReLU, tanh, Sigmoid.

    • It is used to map the non-normalized output of a network to a


    probability distribution over predicted output class. That is it
    converts output of last layer into a essential probability
    distribution.
    Implementation of NN
    Feed Forward
    • Set of Input features and random weights
    • Weights will be optimized by back propagation

    Back Propagation
    • Calculating error between predicted output and target output
    and use Gradient descent method to update weights

    Gradient Descent
    • Machine Learning algorithm
    • It operates iteratively to find the optimal values for its parameters.
    user-defined learning rate, and initial parameter values
    Vanishing & Exploding Gradient
    • It is very common problem in every Neural Network, which is associated
    with Backpropagation.

    • Weights of network are updated through backpropagation by finding


    gradients.

    • When the number of hidden layer is high, then the gradient vanishes or
    explodes as it propagates backward. It leads instability in network,
    unable to learn from training

    • The explosion occurs through exponential growth by repeatedly


    multiplying gradients through the network layers that have values larger
    than 1.0

    • It can be fixed by redesigning the network, using Long Short Term


    Memory networks, Gradient clipping, etc.
    Keras Basic Syntax

    Adding Layers

    model.add(Dense(12, input_dim=8, init='uniform', activation='relu'))

    model.add(Dense(8, activation='relu'))

    Compile Model
    model.compile(loss='binary_crossentropy',
    optimizer='adam', metrics=['accuracy'])
    Batch vs Epoch
    Training occurs over epochs and each epoch is split into batches.

    Epoch - One pass through all of the rows in the training dataset

    Batch - One or more samples considered by the model within an epoch before weights are updated.

    Keras Basic Syntax


    Save & Load Model
    Save Load
    model_json = model.to_json() json_file = open('model.json', 'r')
    with open("model.json", "w") as json_file: loaded_model_json = json_file.read()
    json_file.write(model_json) json_file.close()
    loaded_model = model_from_json(loaded_model_json)
    model.save_weights("model.h5") loaded_model.load_weights("model.h5")
    print("Saved model to disk")
    THANK YOU

    NOVITECH COIMBATORE
    novitechresearchanddevelopment

    You might also like