CHAPTER TWO

2. CONDITIONAL PROBABILITY AND INDEPENDENCE

2.1. Conditional Probability

2.2 Multiplication theorem, Bayes’ Theorem and Total probability theorem

2.3 Independent Events

2.1. Conditional Probability

It is to be recalled that up to this moment we were talking about events that are independent. In
the real world variables are known to be interrelated or dependent to one another, and so do
events. Accordingly, in most cases it seems rare to find probabilities of events that do not involve
conditional probabilities.

The conditional probability will help us calculate the probability of events when it is known
that another event has occurred. In particular, if it is known that some event has already occurred,
then we will find the probability of another event. Conditional probability refers to those
probabilities that arise due to dependence among events. To introduce this important concept , let us
consider the following example.

Example: A box of 80 candles consists of 30 defective and 50 non-defective candles. Suppose

that we choose two candles from this box one after the other.

a) with replacement b) without replacement

Define two events A and B as follows

A={the first candle is defective} B={the second candle is defective}

If we are choosing with replacement, P(A) =P(B)= 30/80 =3/8 because each time the selected
candle will be replaced back and hence the composition remains the same.

However, if we are choosing without replacement, the results are not quite immediate.

P(A)=3/8 but what about P(B)=? It is clear that P(B) depends on whether A did occur or didn’t
occur. If A did not occur then P(B)=30/79. On the other hand, if A did occur then on the second
draw there are only 79 candles left of which 29 are defective. In this case, P(B)= P(B|A)=29/79.
P(B|A) is the conditional probability of B given that A has already occurred.

Page 1 of 6
Definition: Conditional Probability: Let A and B be two events in the sample space S with
P(B) > 0. Then the probability that event A will occur given that event B has already occurred is
called the conditional probability of A given B. It is denoted by P(A|B).

( ∩ )
( |/ ) = ; ℎ ( ) >0
( )

Example: Consider the above example when selection is made without replacement

Remark:

1. ( | ) for fixed B satisfies all of the various axioms of probability.

I. 0 ( | ) 1
II. ( | ) = 1; where S denotes the Sample Space.
III. (( ∪ )| ) = ( | ) + ( | ); where are mutually exclusive
events
IV. If , … , , are a sequence of mutually exclusive events in S, then

( | )= ( | )

2. If the possible outcomes are finitely many and equally likely, we have
∩ n( ∩ )
(A| B) = =
Number of elements of B ( )

(i.e., conditional probability can also be viewed as a probability law on a new universe B,
because all of the conditional probability is concentrated on B)

3. For any events in S:

(( ∪ )| ) = ( | ) + ( | ) − (( ∩ )| )
4. If are events in S and ⊆ then ( | ) ≤ ( | )
5. If is an event in S, then ( | ) = 1 − ( | )

Example 1: Toss a fair coin three times successively. Let

={ ℎ ℎ }

={ ℎ }

Then compute ( | ).

Example 2: For a student enrolling in a freshman at a university, the probability that he/she
will get a scholarship is 0.25 and the probability that he/she will graduate is 0.75. If the

Page 2 of 6
probability that he/she will get scholarship and will also graduate is 0.2. What is the probability
that a student who secured a scholarhip will graduate?

Example 3: Let the probability that a policy will be correctly formulated is 0.60 and the probability
that a policy will be correctly formulated and correctly executed is 0.54. Then find the
probability that:

a. the policy will be correctly executed given that it is correctly formulated

b. The policy will not be correctly executed given that it is correctly formulated.

Example 4: A lot consists of 20 defective and 80 non-defective items from which two items are
chosen without replacement. Events A & B are defined as

A = {the first item chosen is defective}, B = {the second item chosen is defective}

a. What is the probability that both items are defective?

b. What is the probability that the second item is defective?

2.2. Multiplication Theorem, Bayes’ Theorem and Total Probability Theorem

2.2.1. Multiplication Theorem of Probability

Definition: If A and B are two events in S and ( ) is not zero, then the probability that A and
B will both occur is the product of the probability of A and the conditional probability of B given
A. i.e. ( ∩ ) = ( ) ( | ). Similarly, ( ∩ ) = ( ) ( | ).

Multiplication rule of probability is used to find the probability of simultaneous occurrence of

events.

The above mult iplicat ion rule can be generalized to more than two events in the following way.
n
Let A1, A2, …, An be a sequence o f events and assume A
i 1
i   . Then

n
P( A i )  P(A1 ) P(A 2 |A 1 ) P(A 3 |A 1 , A 2 ) .. P(A n |A 1 , A 2 , ., A n -1 )
i 1

Example 1: Suppose that there is a box containing 20 fuses of which 5 are defective. If 3 fuses are
selected at random and removed successively, what is the probability that all are defective?

Example 2: Three balls are drawn at random one after the other without replacement from a box
containing 3 white and 4 red balls. Find the probability that the balls alternate in color.

Page 3 of 6
Example 3: Suppose that our sample space is the population of adults in a small town who have
completed the requirements for a college degree. They are categorized according to sex and
employment status as follows.

Employed Unemployed Total

Male 30 40 70

Female 20 70 90

Total 50 110 160

A person is selected at random. What is the probability that the selected person is

a) a man given that he is employed b)unemployed given that she is female

c) a male or a female d)unemployed male e)employed

2.2.2. Bayes’ Theorem

Definition (Partition of Sample Space): We say that the events B1, B2, ….Bk represent s partit ion
of the sample space S if

a) Bi  Bj = i.e. if the events are mutually exclusive

k
b) B
i 1
i S

c) P(Bi) > 0 i, i = 1, 2, 3, . . . , k

Thus the events B1, B2, ….Bk are mutually exclusive and exhaust ive. Graphically:

Example: Let = {1, 2, 3, 4, 5, 6}. = {1, 2}, = {3, 4, 5} , = {6}

Let A be so me event associated with S and let B1, B2, … Bk be partit ions of S. Hence we can
write

A = (A  B1)  (A B2)  … (A  Bk)

Page 4 of 6
Of course, some of A  Bi may be empty. Note that since B's are partitions and have no
intersection (mutually exclusive events) all sets (A  B1), (A B2), …,(A  Bk) are pair-wise
mutually exclusive.

Hence P(A) = P(A  B1) + P(A B2) + … + P(A  Bk)

But P(A  Bi), for each and every i, may be expressed as P(A|Bi) xP(Bi) and we obtain the theorem
of total probability.

Total Probability Theorem

If we have B1, B2, ….Bk to be partitions of S then for any event A associated with S

P(A) = P(A|B 1) P(B1) + P(A|B 2) P(B2) + …. + P(A|B k) P(Bk)

k
=  P(A|B ) P(B )
i 1
i i

Graphically:

Example: A coin, is weighted so that P(H) = and P(T) = , is tossed. If head appears, then a
number is selected at random from the numbers 1 through 9; if tail appears, then a number is
selected at random from the numbers 1 through 5. Find the probability that an even number is
selected.

Example: Three machines A, B and C produce 50%, 30% and 20% of the total output
respectively. The percentages of defective output of these machines are 3%, 4% and 5%
respectively. If an item is selected at random, what is the probability that the item is defective?

The above example may motivate us to pask oseoon

ne question. Suppose one item is chosen and is
found to be defective. What is the probability that it is from machine A? The fo llowing well-
known theorem, called Bayes’ Theorem, provides the answer!

Bayes’ Theorem: Let B1, B2, …., Bk be a partit ion o f the sample space S and let A be an event
associated with S. Then

Page 5 of 6
 P ( A | Bi ) P ( Bi )
P ( Bi | A)  k

 P( A | B ) P( B )
j 1
j j

Example: Consider the pr evious exa mple. An item wa s selected and is found to be defective. What is the
probability that it is from B1?

In other words, we divide the pr obability of the required path, those paths which lea d to a defective item,
by the probability of the reduced sa mple space.

2.3. Independent events: Two events A and B are said to be independent iff

(  )= ( ) ( )

Verbally speaking, A and B are independent provided knowledge of the occurrence of A by no mea ns
influences the probability of occurrence of B.
Remarks
1) If A and B are mutually exclusive when do they become independent?
P(A  B) = P()  P(A) P(B) = 0
 A and B are independent and mutually exclusive iff A  B =  and either P(A) =0 or P(B)=0
2) If A  B then A  B = A
 P(A  B) = P(A)

 A and B cannot be independent unless P(B)= 1

B=S

3)  and S are independent of any event

4) Three events A, B, and C are jointly independent iff
i. ( ∩ )= ( ) ( )
ii. ( ∩ )= ( ) ( )
iii. ( ∩ )= ( ) ( )
iv. ( ∩ ∩ )= ( ) ( ) ( )
Three events A1, A2, and A3 are said to be pair-wise independent if P(Ai n Aj) = P(Ai)P(Aj). However,
pair-wise independence is not a sufficient condition for the mutual independence of any three events. In general,
for any n events to be jointly independent 2 - n - 1 independence conditions have to be satisfied.

Theorem: Let A and B be independent. Then

I. A and B/ are independent

II. A/ a nd B/ are independent
III. A/ a nd B are independent

Proof: Exercise

Page 6 of 6