UNIT 7 EVALUATION

WORKSHEET
SECTION A (Objective Type Questions)
A Tick (✔) the correct option.
Solved Questions
1. Which of the following is true for Accuracy?
a. It is defined as the percentage of correct predictions out of all the
observations.
b. It is defined as the percentage of true positive cases versus all the cases
where the prediction is true.
c. It can be defined as the fraction of positive cases that are correctly
identified.
d. None of the above
2. What is the formula for Accuracy?
a. Accuracy=((TP + TN)/(TP+TN+FP+FN))*100%
b. Accuracy=( (FP+TN)/(TP+TN+FP+FN))*100%
c. Accuracy =((FP+FN)/(TP+TN+FP+FN))*100%
d. Accuracy=(TP+FN)(TP+TN+FP+FN)*100%
3. With the help of which of the following, Prediction and Reality can be easily
mapped together?
a. Predictions b. Reality c. Confusion Matrix d. Recall
4. What will be the outcome, if the Prediction is "Yes" and it matches with the
Reality? and what will be the outcome, if the Prediction is "Yes" and it does not
match with the Reality?
a. True Negative, False Negative
b. True Negative, False Positive
c. True Positive, False Positive
d. True Positive, True Negative
5. Which of the following is not an example of an evaluation method?
a. Precision b. Accuracy c. Prediction d. Recall
6. _____________is defined as the fraction of positive cases that are correctly
identified.
a. Precision b. Accuracy c. Recall d. F1 Score
7. F1 Score is 1 when:
a. Recall and Precision is 100%
b. Recall is 100%
c. Precision
d. Recall and Precision is 0
8._________is defined as the percentage of correct predictions out of all the
observations.
a. Predictions b. Accuracy c. Reality d. F1 Score
9. Recall-Evaluation method is:
a. defined as the fraction of positive cases that are correctly identified.
b. defined as the percentage of true positive cases versus all the cases
where the prediction is true.
c. defined as the percentage of correct predictions out of all the
observations.
d. comparison between the prediction and reality
10. In a confusion matrix handling data for a model predicting breast cancer,
what does False Positive refer to?
a. Model predicted positive and the person has breast cancer in reality.
b. Model predicted positive and the person does not have breast cancer in
reality.
c. Model predicted negative and the person has breast cancer in reality.
d. Model predicted negative and the person does not have breast cancer in
reality.

B. Fill in the blanks.

1._____________ is defined as the percentage of correct predictions out
of all the observations.
2. False Positive means the predicted value was ___________________.
3. A model is said to have a good performance if the F1 Score for that
model is_______________.
4._______________ is the measure of a test's accuracy.
5.______________is the stage of testing the model.
6. Unauthenticated data can be data collected from.______________
resources.
7._______________is the real scenario of the situation.
8. When the prediction matches with the reality this condition is
called.__________.

C. State whether these statements are true or false.

1. Model Evaluation is not an important stage in AI Model.
2. The lack of training data could lead to inefficiency of the AI Model.
3. The testing data and the training data should be totally different.
4. Prediction and Reality are two important parameters for Model
Evaluation.
5. False Negative is also called Type 1 Error.
D. Match the following:
1. Reality a. Error Matrix
2. Prediction b. Actual Values
3. Confusion Matrix c. Model Evaluation method
4. Columns in Confusion Matrix d. Model Evaluation Parameter
5. False Positive e. Type 1 error
A. Short answer type questions:
SECTION B (Subjective Type Questions)
1. What is Model Evaluation? Why it is important to evaluate a model?
Ans. Model Evaluation is the last stage of the AI Project development cycle. It
is the stage of testing the model where testing data is given to the system and
the output generated is evaluated with the actual result to see the accuracy of the
output and the reliability of the AI model. It is important to evaluate the model
to see that the model is designed as per the need and is giving the desired
process.
2. Give an example where high Accuracy is not useful?
Ans. A robot programmed to cap 2000 bottles in a factory has 99.99% accuracy.
If there occur some errors in the robot due to any reason then this will create a
huge loss in the factory.
3. Give an example where High Precision is not useful?
Ans. Too many False Negatives will make the spam filter ineffective but False
Positives may cause important mails to be missed and hence Precision is not
usable.
4. Create a confusion matrix for the following problem?
Let us imagine that we have an AI model which identifies an apple from a
mango. Following are the cases
Predicted Reality
Yes No
Yes No
Yes Yes
No Yes
No Yes
Yes Yes
No No
Ans. The Confusion Matrix
Confusion matrix Reality
YES NO
YES True Positive False Positive
Prediction NO False Negative True Negative
5. List down the importance of the Evaluation Process?
Ans.
● Evaluation ensures that the model is operating correctly and optimally.
● Evaluation is an initiative to understand how well it achieves its goals.
● Evaluations help to determine what works well and what could be
improved in a program
B. Long answer type questions:
1. Differentiate between Prediction and Reality.
Ans. Following are the differences between Prediction and Reality:
a. Prediction is the input given to the machine to receive the expected result of
reality.
b. Prediction is the output given to match reality.
c. The prediction is the output which is given by the machine and the reality is
the real scenario in which the prediction has been made.
d. Prediction and reality both can be used interchangeably.
2. Which evaluation metric would be crucial in the following cases? Justify your
answer.
a. Mail Spam
b. Gold mining
c. Viral Outbreak
Ans. a. If the model always predicts that the mail is spam, people would not
look at it and eventually might lose important information. False Positive
condition would have a high cost.
b. A model saying that there exists treasure at a point and you keep on digging
there but it turns out that it is a false alarm. False Positive case is very costly.
c. A deadly virus has started spreading and the model which is supposed to
predict a viral outbreak does not detect it. The virus might spread widely and
infect a lot of people. Hence, False Negative can be dangerous.
3. Select the correct category for each scenario.
1. The AI model predicted that the patient admitted is
Covid Positive, and RTPCR test is positive TP TN FP FN
2. The AI model predicted that the patient admitted is
not Covid Positive, and RTPCR test is negative TP TN FP FN
3. The AI model predicted that the patient admitted is
Covid Positive, and RTPCR test is negative TP TN FP FN
4. The AI model predicted that the patient admitted is
not Covid Positive, and RTPCR test is Positive TP TN FP FN
Storms are frequently experienced all over the world. Storms cause severe
damage to people and property. Imagine that you have come up with an AI
based prediction model that predicts whether there will be storms in a given
region or not. Confusion Matrix for the same is:
Confusion Matrix Actual:1 Actual: 0
Predicted 1 6 3
Predicted 0 20 65
Calculate Accuracy, Precision, Recall and F1 Score
Ans.
Accuracy =((TP + TN) /(TP+TN+FP+FN))* 100%
6+65/(6+65+3+20)*100%
71/94*100=75.5%
Precision =(TP/TP+FP)*100
(6/6+3)*100=66.66%
Recall=(TP/TP+FN)=0.23
F1 Score = 2 *(Precision * Recall/Precision + Recall)
2*(66.66 * 0.23/66.66 +0.23)
2*(15.33/66.89)= 0.45
4. Give 3 reasons for AI Model inefficiency?
Ans. Lack of Training Data: The lack of training data could be due to less data
available for developing an AI Model, or the data is missed while training the
model then the AI model created will not be efficient.
Unauthenticated Data / Wrong Data: If the data is not authenticated and correct
due to negligence or data collected from unauthorized resources then the model
will not give good results.
Inefficient coding / Wrong Algorithms: If the written algorithms are not correct
and relevant, Model will not give desired output.
C. Competency-based/Application-based questions:
1. An Al model made the following predictions for Book Sales forecast.
Calculate Accuracy, precision and recall for the following confusion matrix:
Confusion Matrix True Positives True Negatives
Predicted Positive 50 40
Predicted Negative 12 10
Ans. TP 50 40 FP
FN 12 10 TN
Accuracy =Correction prediction/Total Cases
=((TP + TN)/ (TP + TN+FP+ FN))*100%
((50+10)/(50+ 10+40 + 12))*100%
(60/112)*100% = 55.5%
Precision= True Positive/ All Predicted Positive*100%
=(TP/ TP + FP)* 100%
=(50/50 + 40)* 100% = 55.5%
Recall = (True Positive/ True Positive + False Negative)* 100%
=50/50+ 12= 0.80
2. Assertion: Evaluation is the process of understanding the outcome of any AI
model.
Reason: There can be different Evaluation techniques, depending on the type
and purpose of the model.
(i) Both A and R are correct and R is the correct explanation of A.
(ii) Both A and R are correct but R is NOT the correct explanation of A.
(iii) A is correct but R is incorrect.
(iv) A is incorrect but R is correct.
Ans. (iv) A is incorrect but R is correct.

Key
Answers
Exercise (Section A)
A. 1. a
2. a
3. c
4. c
5. c
6. c
7. a
8. b
9. a
10. b

B. 1. Accuracy
2. falsely predicted
3. high
4. F-measure
5. Model Evaluation
6. unauthorized
7. Reality
8. True Positive

C. 1. False 2. True 3. True 4. True 5. False

D. 1. d 2. c. 3. a 4. b 5. e
Unsolved Questions
Tick (✔) the correct option:
SECTION A (Objective Type Questions)
1. An AI model is trained to predict the traffic jams in a particular highway.
What according to you can be the data features for such a type of model?
a. Average traffic on weekdays
b. Day and time of traffic jams
c. Weather conditions on the days of traffic jams
d. All of the above
2. If prediction is low and recall is high, what will be the F1 score?
a. Same b. Low c. High d. None
3. Which of the following is true when the value of precision is low?
a. True Positive is high b. False Positive is low
c. False Positive is high d. None of the above
4. What will be the outcome, if the Prediction is "Yes" and it matches with the
Reality? What will be the outcome, if the Prediction is "Yes" and it does not
match the Reality?
a. True Positive, True Negative. b. True Negative, False Negative
c. True Negative, False Positive d. True Positive, False Positive
5. Differentiate between Prediction and Reality.
a. Prediction is the input given to the machine to receive the expected
result of the reality.
b. Prediction is the output given to match the reality.
c. The prediction is the output which is given by the machine and the
reality is the real scenario in which the prediction has been made.
d. Prediction and reality both can be used interchangeably.
6. Which of the following statements is true for the term Evaluation?
a. Helps in classifying the type and genre of a document.
b. It helps in predicting the topic for a corpus.
c. Helps in understanding the reliability of any Al model.
d. Process to extract the important information out of a corpus.
7. Prediction and Reality can be easily mapped together with the help of:
a. Prediction b. Reality c. Accuracy d. Confusion Matrix
8. You have 3 AI models solving the same problem. How will you choose the
best model?
a. The model with the highest accuracy is the best.
 b. The model with the best F1 score is the best.
c. The model with the highest precision is the best.
d. The model with the highest recall is the best.
B. Fill in the blanks.
1._____________can be defined as the measure of balance between
precision and recall.
2. When the prediction is No and the reality is Yes then this condition is
called as __________.
3.__________ is a table made with 4 different combinations of predicted
and actual values in the form of a 2X2 matrix.
4. In the Confusion Matrix the target variable has two values
__________and ____________.
5. Another name for False Positive is __________________.
6. High Accuracy means _____________ performance of the AI model.
7. A prediction is said to be correct if it matches with ___________.
8.___________is the percentage of True Positive cases and All Predicted
Positive Cases.
C. Match the following:
1. False Negative a. Error Matrix
2. False Positive b. Sensitivity
3. Recall c. F-measure
4. Confusion Matrix d. Actual Value Positive
5. F1 score e. Actual Value Negative
f. Actual Value zero
D. State whether these statements are true or false.
1. A good F1 score means you have low false positives and low false
negatives.
2. Another name for recall is precision.
3. False Negative or False Positive have no effect on the cost.
4. Both Recall and Precision are important for finding the performance of
the AI Model.
5. Confusion matrix is not useful for measuring recall.
SECTION B (Subjective Type Questions)
A. Short answer type questions:
1. Give one use of the Confusion matrix.
2. What is Recall? Give its formula.
3. Explain in short the important terminologies of Confusion Matrix.
4. Give two reasons for the inefficiency of the AI Model.
B. Long answer type questions:
1. A Model made the following predictions. Calculate Accuracy, Precision,
Recall and F1 Score.
Confusion Matrix Reality 1 Reality 0
Predicted 1 100 85
Predicted 0 80 32
Ans. (Calculate by yourself)
2. In today's Scenario, People receive constant mails from the organizations they
are connected with and they have to be classified into spam and non-spam as
required. Now it has to be decided which mails to be categorized and set as
spam mails according to the mails received till date, The confusion matrix for
the same is as follows:
Confusion Matrix Reality 1 Reality 0
Predicted 1 40 24
Predicted 0 10 36
Give an example for:
a. High False Negative cost
b. High False Positive cost
Ans. a. High False Negative cost: In a case like forest fire, a False Negative can
cost us a lot and is risky too. Imagine no alert being given even when there is
Forest Fire. The whole forest might burn down.
Another case where a False Negative can be dangerous is Viral Outbreak.
Imagine a deadly virus has started spreading and the model which is supposed
to predict a viral outbreak does not detect it. The virus might spread widely and
infect a lot of people.
b. High False Positive cost: In case of Mining, imagine a model telling you that
there exists treasure at a point and you keep on digging there but it turns out that
it is a false alarm. Here, False Positive case (predicting there is treasure but
there is no treasure) can be very costly.
Similarly, consider a model that predicts that a mail is spam or not. If the model
always predicts that the mail is spam, people would not look at it and eventually
might lose important information. Here also False Positive condition
(Predicting the mail as spam while the mail is not spam) would have a high cost.
3. In the state there is a spread of dengue, and precautions are to be taken to stop
the spread, so that medicine stocks can be arranged, some actual cases have
been detected and the frequency of the infections have been detected, following
predictions are being done, the confusion matrix for the same is as follows:
Confusion Matrix Reality 1 Reality 0
Predicted 1 24 14
Predicted 0 8 28
Why is F1 score considered the best of all the Evaluation methods?
Ans. A good F1 score has low false positives and low false negatives, so you’re
correctly identifying real threats, and you are not disturbed by false alarms. It is
the measure of balance between precision and recall. The F1 score is a number
between 0 and 1 and is the harmonic mean of precision and recall. A model is
said to have a good performance if the F1 Score for that model is high.
4. Differentiate between True Positive and False Positive.
Ans.
● True Positive: The Predicted value matches the actual value i.e., the
actual value was positive and the model predicted a positive value.
● False Positive (Type 1 error): The predicted value was falsely predicted
i.e., the actual value was negative but the model predicted a positive
value.

Key
UNSOLVED QUESTIONS
Exercise (Section A)
A. 1. a
2. b
3. c
4. d
5. c
6. c
7. d
8. b
B. 1. F1 Score
2. False Negative
3. Confusion Matrix
4. Positive, Negative
5. Type 1 error
6. good
7. Reality
8. Precision
C. 1. d 2. e 3. b 4. a 5. c
D. 1. False 2. False 3. False 4. True 5. False

SECTION B (Subjective Type Questions)

A. Short answer type questions:
1. Give one use of the Confusion matrix.
Ans. Confusion Matrix helps in measuring the performance of an AI model
using the test data.
2. What is Recall? Give its formula.
Ans. Recall is defined as the fraction of positive cases that are correctly
identified. It majorly takes into account the true reality cases i.e., it is a measure
of our model correctly identifying True Positives.
Recall=TP/TP+FN
3. Explain in short the important terminologies of Confusion Matrix.
Ans. Important terminologies of Confusion Matrix are:
● Positive: The prediction is positive for the scenario.
● Negative: The prediction is negative for the scenario.
● True Positive: The Predicted value matches the actual value i.e., the
actual value was positive and the model predicted a positive value.
● True Negative: The Predicted value matches the actual value i.e., the
actual value was Negative and the model predicted a Negative value.
● False Positive (Type 1 error): The predicted value was falsely predicted
i.e., the actual value was negative but the model predicted a positive
value.
● False Negative (Type 2 error): The predicted value was falsely predicted
i.e., the actual value was positive but the model predicted a negative
value.
4. Give two reasons for the inefficiency of the AI Model.
Ans. The inefficiency of the AI Model is due to
● Lack of Training Data: The lack of training data could be due to less data
available for developing an AI model, or the data is missed while training
the model then the AI model created will not be efficient.
● Unauthenticated Data/wrong data: If the data is not authenticated and
correct due to negligence or data collected from unauthorized resources
then the model will not give good results.
● Inefficient coding/Wrong Algorithms: If the written algorithms are not
correct and relevant, AI model will not give desired output.
● Not Tested: If the model is not tested properly, then it will not be
efficient.
● Not Easy: If it is not easy to be implemented in production or scalable.
● Less Accuracy: A model is not efficient if it gives less accuracy scores in
production or test data or if it is not able to generalize well on unseen
data.