Dynamic Testing: Black-Box Testing

Techniques
Dr.B.Sathis Kumar
 Black box testing strategies

• Black-Box Testing Techniques- Equivalent

partitioning-Boundary Value Analysis (BVA)-
State Transition Testing-Decision table based
Testing - Cause-Effect Graphing Based Testing -
Error Guessing.
 What is Black Box Testing?

• Black box testing, which is also known as

behavioral, opaque-box, closed-box,
specification-based or eye-to-eye testing, is a
Software Testing method that analyses the
functionality of a software/application
without knowing much about the internal
structure/design of the item that is being
tested and compares the input value with the
output value.
 Black Box Testing
• This testing occurs
throughout the software
development and Testing
life cycle i.e in Unit,
Integration, System,
Acceptance, and
regression testing stages.
• This can be both
functional or non-
functional.
 Types of Black Box Testing

1) Functional Testing
This type deals with the functional requirements
or specifications of an application. Here, different
actions or functions of the system are being tested
by providing the input and comparing the actual
output with the expected output.
• For Example, when we test a Dropdown list, we
click on it and verify that it expands and all the
expected values are showing in the list.
 Few major types of Functional Testing are:

• Smoke Testing
• Sanity Testing
• Integration Testing
• System Testing
• Regression Testing
• User Acceptance Testing
 2) Non-Functional Testing

• Apart from the functionalities of the requirements, there are

several non-functional aspects as well that are required to be
tested to improve the quality and performance of the
application.
• Few major types of Non-functional testing include:
• Usability Testing
• Load Testing
• Performance Testing
• Compatibility Testing
• Stress Testing
• Scalability Testing
 Use of Black box testing
• Black-box testing attempts to find errors in the following
categories:
• To test the modules independently
• To test the functional validity of the software so that
incorrect or missing functions can be recognized
• To look for interface errors
• To test the system behavior and check its performance
• To test the maximum load or stress on the system
• To test the software such that the user/customer accepts
the system within defined acceptable limits
 1) Boundary Value Analysis:

• From the name itself, we can understand that in

this technique we focus on the values at
boundaries as it is found that many applications
have a high amount of issues on the boundaries.
• Boundary means the values near the limit where
the behavior of the system changes.
• In boundary value analysis both the valid inputs
and invalid inputs are being tested to verify the
issues.
• If we want to test a field where values from 1
to 100 should be accepted then we choose
the boundary values: 1-1, 1, 1+1, 100-1, 100,
and 100+1. Instead of using all the values from
1 to 100, we just use 0, 1, 2, 99, 100, and 101.
 BOUNDARY VALUE CHECKING (BVC)

• In this method, the test cases are designed by

holding one variable at its extreme value and other
variables at their nominal values in the input
domain.
The variable at its extreme value can be selected at:
• (a) Minimum value (Min)
• (b) Value just above the minimum value (Min+ )
• (c) Maximum value (Max)
• (d) Value just below the maximum value (Max−)
• Let us take the example of two variables, A
and B. If we consider all the above
combinations with nominal values, then
following test cases can be designed:
 ROBUSTNESS TESTING METHOD

• The idea of BVC can be extended such that

boundary values are exceeded as:
• A value just greater than the Maximum value
(Max+)
• A value just less than Minimum value (Min−)
• When test cases are designed considering the
above points in addition to BVC, it is called
robustness testing.
• Let us take the previous example again. Add
the following test cases to the list of 9 test
cases designed in BVC:
 WORST-CASE TESTING METHOD
• We can again extend the concept of BVC by
assuming more than one variable on the
boundary. It is called worst-case testing
method.
• Again, take the previous example of two
variables, A and B.
• We can add the following test cases to the list
of 9 test cases designed in BVC as:
 Exercise
• A program reads an integer number within the
range [1,100] and determines whether it is a
prime number or not. Design test cases for
this program using BVC, robust testing, and
worst-case testing methods
• Test cases using BVC Since there is one
variable, the total number of test cases will be
4n + 1 = 5.
• In our example, the set of minimum and
maximum values is shown below:
 2) Equivalence Partitioning:

• This technique is also known as Equivalence Class

Partitioning (ECP). In this technique, input values to the
system or application are divided into different classes
or groups based on its similarity in the outcome.
• Hence, instead of using each and every input value we
can now use any one value from the group/class to test
the outcome.
• In this way, we can maintain the test coverage while
we can reduce a lot of rework and most importantly
the time spent.
 Two invalid classes will be:

a) Less than or equal to 17.

b) Greater than or equal to 61.
• One valid class will be anything between 18 to
60.
• We have thus reduced the test cases to only 3
test cases based on the formed classes thereby
covering all the possibilities. So, testing with any
one value from each set of the class is sufficient
to test the above scenario
 Equivalence partitioning method for
designing test cases -Goals
• Completeness Without executing all the test cases, we strive to
touch the completeness of testing domain.
• Non-redundancy When the test cases are executed having
inputs from the same class, then there is redundancy in
executing the test cases. Time and resources are wasted in
executing these redundant test cases, as they explore the
same type of bug. Thus, the goal of equivalence partitioning
method is to reduce these redundant test cases.
To use equivalence partitioning, one needs to perform two steps:
1. Identify equivalence classes
2. Design test cases
IDENTIFICATION OF EQUIVALENT CLASSES

Valid equivalence classes These classes consider

valid inputs to the program.
Invalid equivalence classes One must not be
restricted to valid inputs only.
• We should also consider invalid inputs that
will generate error conditions or
• unexpected behaviour of the program
 Equivalence Partitioning
• Partition the test cases into "equivalence classes"

• Each equivalence class contains a set of "equivalent" test cases

• Two test cases are considered to be equivalent if we expect

the program to process them both in the same way (i.e., follow
the same path through the code)

• If you expect the program to process two test cases in the

same way, only test one of them, thus reducing the number of
test cases you have to run
 Equivalence Partitioning

• First-level partitioning: Valid vs. Invalid test cases

Valid Invalid
 Equivalence Partitioning

• Partition valid and invalid test cases into equivalence

classes
 Equivalence Partitioning

• Create a test case for at least one value from each

equivalence class
 Equivalence Partitioning

• When designing test cases, you may use different

definitions of “equivalence”, each of which will
partition the test case space differently
– Example: int Add(n1, n2, n3, …)
• Equivalence Definition 1: partition test cases by the number of
inputs (1, 2, 3, etc.)
• Equivalence Definition 2: partition test cases by the number
signs they contain (positive, negative, both)
• Equivalence Definition 3: partition test cases by the magnitude
of operands (large numbers, small numbers, both)
• Etc.
 Equivalence Partitioning

• When designing test cases, you may use different

definitions of “equivalence”, each of which will
partition the test case space differently
– Example: string Fetch(URL)
• Equivalence Definition 1: partition test cases by URL protocol
(“http”, “https”, “ftp”, “file”, etc.)
• Equivalence Definition 2: partition test cases by type of file
being retrieved (HTML, GIF, JPEG, Plain Text, etc.)
• Equivalence Definition 3: partition test cases by length of URL
(very short, short, medium, long, very long, etc.)
• Etc.
Equivalence Partitioning - examples

Input Valid Equivalence Classes Invalid Equivalence Classes

? ?
A integer N such that:
-99 <= N <= 99

? ?
Phone Number
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
Equivalence Partitioning - examples

Input Valid Equivalence Classes Invalid Equivalence Classes

?
A integer N such that: [-99, -10]
-99 <= N <= 99 [-9, -1]
0
[1, 9]
[10, 99]

? ?
Phone Number
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
Equivalence Partitioning - examples

Input Valid Equivalence Classes Invalid Equivalence Classes

A integer N such that: [-99, -10] < -99
-99 <= N <= 99 [-9, -1] > 99
0 Malformed numbers
[1, 9] {12-, 1-2-3, …}
[10, 99] Non-numeric strings
{junk, 1E2, $13}
Empty value
Phone Number
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
? ?
Equivalence Partitioning - examples

Input Valid Equivalence Classes Invalid Equivalence Classes

A integer N such that: [-99, -10] < -99
-99 <= N <= 99 [-9, -1] > 99
0 Malformed numbers
[1, 9] {12-, 1-2-3, …}
[10, 99] Non-numeric strings
{junk, 1E2, $13}
Empty value
Phone Number 555-5555
Area code: [200, 999]
Prefix: (200, 999]
Suffix: Any 4 digits
(555)555-5555
555-555-5555
200 <= Area code <= 999
?
200 < Prefix <= 999
Equivalence Partitioning - examples

Input Valid Equivalence Classes Invalid Equivalence Classes

A integer N such that: [-99, -10] < -99
-99 <= N <= 99 [-9, -1] > 99
0 Malformed numbers
[1, 9] {12-, 1-2-3, …}
[10, 99] Non-numeric strings
{junk, 1E2, $13}
Empty value
Phone Number 555-5555 Invalid format 5555555,
Area code: [200, 999] (555)555-5555 (555)(555)5555, etc.
Prefix: (200, 999] 555-555-5555 Area code < 200 or > 999
Suffix: Any 4 digits 200 <= Area code <= 999 Area code with non-numeric
200 < Prefix <= 999 characters
Similar for Prefix and Suffix
Guidelines are defined for forming
equivalence classes:
• If a program handles each valid input
differently, then define one valid equivalence
class per valid input
• Boundary value analysis can help in identifying
the classes. For example, for an input condition,
say 0 <= a <= 100, one valid equivalent class can
be formed from the valid range of a. And with
BVA, two invalid classes that cross the minimum
and maximum values can be identified,
• i.e. a < 0 and a > 100.
• If an input variable can identify more than
one category, then for each category, we can
make equivalent classes.
• For example, if the input is a character, then it
can be an alphabet, a number, or a special
character.
• So we can make three valid classes for this
input and one invalid class
 IDENTIFYING THE TEST CASES

• A few guidelines are given below to identify test

cases through generated equivalence classes:
• Assign a unique identification number to each
equivalence class.
• Write a new test case covering as many of the
uncovered valid equivalence classes as possible,
until all valid equivalence classes have been
covered by test cases.
• A program reads three numbers, A, B, and C,
with a range [1, 50] and prints the largest
number. Design test cases for this program
using equivalence class testing technique.
 3) Decision Table Testing:

• As the name itself suggests that, wherever there are logical

relationships like:
• If
{
(Condition = True)
then action1 ;
}
else action2; /*(condition = False)*/
• Then a tester will identify two outputs (action1 and action2) for
two conditions (True and False). So based on the probable
scenarios a Decision table is carved to prepare a set of test
cases.
 For Example:

• Take an example of XYZ bank that provides

interest rate for the Male senior citizen as 10%
and for rest of the people 9%.
 For Example:

• Take an example of XYZ bank that provides

interest rate for the Male senior citizen as 10%
and for rest of the people 9%.
 #4) State Transition Testing:

• State Transition Testing is a technique that is used to

test the different states of the system under test. The
state of the system changes depending upon the
conditions or events. The events trigger states which
become scenarios and a tester needs to test them.
• A systematic state transition diagram gives a clear
view of the state changes but it is effective for
simpler applications. More complex projects may
lead to more complex transition diagrams thus
making it less effective.
 Cause-Effect Graphing Based Testing
• boundary value analysis and equivalence class
partitioning methods do not consider combinations of
input conditions.
• Decision tables, cause-effect graphing is another
technique for combinations of input conditions.
• But cause-effect graphing takes the help of decision
table to design a test case.
• Therefore, cause-effect graphing is the technique to
represent the situations of combinations of input
conditions and then we convert the cause effect graph
into decision table for the test cases.
 Cause-Effect Graphing
• Cause-Effect Graphing is a technique which
starts with a set of requirements and
determines the minimum possible test cases
for maximum test coverage which reduces test
execution time and ultimately cost.
Let’s draw a cause and effect graph based on a situation

Situation:
• The “Print message” is software that read two characters and,
depending on their values, messages must be printed.
• The first character must be an “A” or a “B”.
• The second character must be a digit.
• If the first character is an “A” or “B” and the second character is
a digit, the file must be updated.
• If the first character is incorrect (not an “A” or “B”), the
message X must be printed.
• If the second character is incorrect (not a digit), the message Y
must be printed.
• Solution:
• The causes for this situation are:
C1 – First character is A
C2 – First character is B
C3 – the Second character is a digit
• The effects (results) for this situation are
E1 – Update the file
E2 – Print message “X”
E3 – Print message “Y”
 LET’S START!!

• First, draw the causes and effects as shown

below:
• In this example, let’s start with Effect E1.
• Effect E1 is to update the file. The file is
updated when
–  The first character is “A” and the second
character is a digit
–  The first character is “B” and the second
character is a digit
–  The first character can either be “A” or “B”
and cannot be both.
• Now let’s put these 3 points in symbolic form:
• For E1 to be true – following are the causes:
–  C1 and C3 should be true
–  C2 and C3 should be true
–  C1 and C2 cannot be true together. This
means C1 and C2 are mutually exclusive.
• So as per the above diagram, for E1 to be true
the condition is
• The circle in the middle is
just an interpretation of the
middle point to make the
graph less messy.
There is a third condition
where C1 and C2 are
mutually exclusive. So the
final graph for effect E1 to
be true is shown
Effect E2:
• E2 states to print
message “X”. Message X
will be printed when the
First character is neither
A nor B.
• Which means Effect E2
will hold true when
either C1 OR C2 is invalid.
• So the graph for Effect E2
is shown as (In blue line)
Effect E3
• E3 states to print
message “Y”. Message Y
will be printed when
Second character is
incorrect.
• Which means Effect E3
will hold true when C3
is invalid. So the graph
for Effect E3 is shown as
(In Green line)
 Writing Decision table based on Cause and Effect graph

• First, write down the

Causes and Effects in a
single column shown
below
For E3 to be true, C3 should be false.
• Let’s complete the graph by adding 0 in the
blank column and including the test case
identifier.
 Writing Test cases from the decision table

• I am writing a sample test case for test case 1

(TC1) and Test Case 2 (TC2).
 Error guessing Technique

• To design test cases based on error guessing technique, the

analyst can use the past experiences to identify the conditions.
This technique can be used at any level of testing and for
testing the common mistakes like:
• Divide by zero
• Entering blank spaces in the text fields
• Pressing the submit button without entering values.
• Uploading files exceeding maximum limits.
• Null pointer exception.
• Invalid parameters
Purpose of Error guessing in software testing

• The main purpose of this technique is to make

an educated guess about possible bugs in the
areas where formal testing would not work.
• It should obtain an all-inclusive set of testing
without any skipped areas, and without creating
redundant tests.
• This technique compensates for the
characteristic incompleteness of boundary value
analysis and equivalence portioning techniques.
 Factors used to Guess the Errors

• Error guessing technique requires skilled and

experienced tester. It is mainly based on
intuition and experience.
Following factors can be used to guess the errors:

• Lessons learned from past releases

• Tester’s intuition
• Historical learning
• Previous defects
• Production tickets
• Review checklist
• Application UI
• Previous test results
• Risk reports of the application
• Variety of data used for testing.
• General testing rules
• Knowledge about AUT
 Guidelines for Error guessing

• Remember previously troubled areas: During

any of your testing assignment, whenever you
come across an interesting bug, note it down
for your future reference. There are generally
some common errors that happen in a certain
specific type of applications. Refer to the list of
common errors for the type of application you
are working on.
Improve your technical understanding.
• Check how the code is written and how the concepts like a
null pointer, arrays, indexes, boundaries, loops, etc are
implemented in the code.
• Gain knowledge of the technical environment (server,
operating system, database) in which the application is
hosted.
• Do not just look for errors in the code but also look for errors
and ambiguity in requirements, design, build, testing and
usage.
• Understand the system under testEvaluate historical data and
test resultsKeep awareness of typical implementation errors
 Error Guessing Example

• Suppose there is a requirement stating that the mobile

number should be numeric and not less than 10
characters.
• And, the software application is having a mobile no.
field.
• What will be the result if the mobile no. is left blank?
• What will be the result if any character other than a
numeral is entered?
• What will be the result if less than 10 numerals are
entered?
 Advantages of Error Guessing Technique

• Proves out to be very effective when used in

combination with other formal testing techniques.
• It uncovers those defects which would otherwise be
not possible to find out through formal testing. Thus,
the experience of the tester saves a lot of time and
effort.
• Error guessing supplements the formal test design
techniques.
• Very helpful to guess problematic areas of the
application.
 Drawbacks of Error Guessing Technique

• The focal shortcoming of this technique is that

it is person dependent and thus the
experience of the tester totally controls the
quality of test cases.
• It also cannot guarantee that the software has
reached the expected quality benchmark.
• Only experienced testers can perform this
testing. You can’t get it done by freshers.