Conditional probability distribution

For discrete random variables, the conditional probability mass function of ${\displaystyle Y}$  given ${\displaystyle X=x}$  can be written according to its definition as:

Due to the occurrence of ${\displaystyle P(X=x)}$  in the denominator, this is defined only for non-zero (hence strictly positive) ${\displaystyle P(X=x).}$ 

The relation with the probability distribution of ${\displaystyle X}$  given ${\displaystyle Y}$  is:

${\displaystyle P(Y=y\mid X=x)P(X=x)=P(\{X=x\}\cap \{Y=y\})=P(X=x\mid Y=y)P(Y=y).}$ 

Consider the roll of a fair ‹See Tfd›die and let ${\displaystyle X=1}$  if the number is even (i.e., 2, 4, or 6) and ${\displaystyle X=0}$  otherwise. Furthermore, let ${\displaystyle Y=1}$  if the number is prime (i.e., 2, 3, or 5) and ${\displaystyle Y=0}$  otherwise.

Then the unconditional probability that ${\displaystyle X=1}$  is 3/6 = 1/2 (since there are six possible rolls of the dice, of which three are even), whereas the probability that ${\displaystyle X=1}$  conditional on ${\displaystyle Y=1}$  is 1/3 (since there are three possible prime number rolls—2, 3, and 5—of which one is even).

Similarly for continuous random variables, the conditional probability density function of ${\displaystyle Y}$  given the occurrence of the value ${\displaystyle x}$  of ${\displaystyle X}$  can be written as^{[2]: p. 99}

where ${\displaystyle f_{X,Y}(x,y)}$  gives the joint density of ${\displaystyle X}$  and ${\displaystyle Y}$ , while ${\displaystyle f_{X}(x)}$  gives the marginal density for ${\displaystyle X}$ . Also in this case it is necessary that ${\displaystyle f_{X}(x)>0}$ .

The relation with the probability distribution of ${\displaystyle X}$  given ${\displaystyle Y}$  is given by:

${\displaystyle f_{Y\mid X}(y\mid x)f_{X}(x)=f_{X,Y}(x,y)=f_{X|Y}(x\mid y)f_{Y}(y).}$ 

The concept of the conditional distribution of a continuous random variable is not as intuitive as it might seem: Borel's paradox shows that conditional probability density functions need not be invariant under coordinate transformations.

 

The graph shows a bivariate normal joint density for random variables ${\displaystyle X}$  and ${\displaystyle Y}$ . To see the distribution of ${\displaystyle Y}$  conditional on ${\displaystyle X=70}$ , one can first visualize the line ${\displaystyle X=70}$  in the ${\displaystyle X,Y}$  plane, and then visualize the plane containing that line and perpendicular to the ${\displaystyle X,Y}$  plane. The intersection of that plane with the joint normal density, once rescaled to give unit area under the intersection, is the relevant conditional density of ${\displaystyle Y}$ .

${\displaystyle Y\mid X=70\ \sim \ {\mathcal {N}}\left(\mu _{Y}+{\frac {\sigma _{Y}}{\sigma _{X}}}\rho (70-\mu _{X}),\,(1-\rho ^{2})\sigma _{Y}^{2}\right).}$ 

Random variables ${\displaystyle X}$ , ${\displaystyle Y}$  are independent if and only if the conditional distribution of ${\displaystyle Y}$  given ${\displaystyle X}$  is, for all possible realizations of ${\displaystyle X}$ , equal to the unconditional distribution of ${\displaystyle Y}$ . For discrete random variables this means ${\displaystyle P(Y=y|X=x)=P(Y=y)}$  for all possible ${\displaystyle y}$  and ${\displaystyle x}$  with ${\displaystyle P(X=x)>0}$ . For continuous random variables ${\displaystyle X}$  and ${\displaystyle Y}$ , having a joint density function, it means ${\displaystyle f_{Y}(y|X=x)=f_{Y}(y)}$  for all possible ${\displaystyle y}$  and ${\displaystyle x}$  with ${\displaystyle f_{X}(x)>0}$ .

Seen as a function of ${\displaystyle y}$  for given ${\displaystyle x}$ , ${\displaystyle P(Y=y|X=x)}$  is a probability mass function and so the sum over all ${\displaystyle y}$  (or integral if it is a conditional probability density) is 1. Seen as a function of ${\displaystyle x}$  for given ${\displaystyle y}$ , it is a likelihood function, so that the sum (or integral) over all ${\displaystyle x}$  need not be 1.

Additionally, a marginal of a joint distribution can be expressed as the expectation of the corresponding conditional distribution. For instance, ${\displaystyle p_{X}(x)=E_{Y}[p_{X|Y}(x\ |\ Y)]}$ .

Let ${\displaystyle (\Omega ,{\mathcal {F}},P)}$  be a probability space, ${\displaystyle {\mathcal {G}}\subseteq {\mathcal {F}}}$  a ${\displaystyle \sigma }$ -field in ${\displaystyle {\mathcal {F}}}$ . Given ${\displaystyle A\in {\mathcal {F}}}$ , the Radon-Nikodym theorem implies that there is^[3] a ${\displaystyle {\mathcal {G}}}$ -measurable random variable ${\displaystyle P(A\mid {\mathcal {G}}):\Omega \to \mathbb {R} }$ , called the conditional probability, such that$${\displaystyle \int _{G}P(A\mid {\mathcal {G}})(\omega )dP(\omega )=P(A\cap G)}$$ for every ${\displaystyle G\in {\mathcal {G}}}$ , and such a random variable is uniquely defined up to sets of probability zero. A conditional probability is called regular if ${\displaystyle \operatorname {P} (\cdot \mid {\mathcal {G}})(\omega )}$  is a probability measure on ${\displaystyle (\Omega ,{\mathcal {F}})}$  for all ${\displaystyle \omega \in \Omega }$  a.e.

Special cases:

• For the trivial sigma algebra ${\displaystyle {\mathcal {G}}=\{\emptyset ,\Omega \}}$ , the conditional probability is the constant function ${\displaystyle \operatorname {P} \!\left(A\mid \{\emptyset ,\Omega \}\right)=\operatorname {P} (A).}$ 
• If ${\displaystyle A\in {\mathcal {G}}}$ , then ${\displaystyle \operatorname {P} (A\mid {\mathcal {G}})=1_{A}}$ , the indicator function (defined below).

Let ${\displaystyle X:\Omega \to E}$  be a ${\displaystyle (E,{\mathcal {E}})}$ -valued random variable. For each ${\displaystyle B\in {\mathcal {E}}}$ , define $${\displaystyle \mu _{X\,|\,{\mathcal {G}}}(B\,|\,{\mathcal {G}})=\mathrm {P} (X^{-1}(B)\,|\,{\mathcal {G}}).}$$ For any ${\displaystyle \omega \in \Omega }$ , the function ${\displaystyle \mu _{X\,|{\mathcal {G}}}(\cdot \,|{\mathcal {G}})(\omega ):{\mathcal {E}}\to \mathbb {R} }$  is called the conditional probability distribution of ${\displaystyle X}$  given ${\displaystyle {\mathcal {G}}}$ . If it is a probability measure on ${\displaystyle (E,{\mathcal {E}})}$ , then it is called regular.

For a real-valued random variable (with respect to the Borel ${\displaystyle \sigma }$ -field ${\displaystyle {\mathcal {R}}^{1}}$  on ${\displaystyle \mathbb {R} }$ ), every conditional probability distribution is regular.^[4] In this case,${\displaystyle E[X\mid {\mathcal {G}}]=\int _{-\infty }^{\infty }x\,\mu _{X\mid {\mathcal {G}}}(dx,\cdot )}$  almost surely.

For any event ${\displaystyle A\in {\mathcal {F}}}$ , define the indicator function:

${\displaystyle \mathbf {1} _{A}(\omega )={\begin{cases}1\;&{\text{if }}\omega \in A,\\0\;&{\text{if }}\omega \notin A,\end{cases}}}$ 

which is a random variable. Note that the expectation of this random variable is equal to the probability of A itself:

${\displaystyle \operatorname {E} (\mathbf {1} _{A})=\operatorname {P} (A).\;}$ 

Given a ${\displaystyle \sigma }$ -field ${\displaystyle {\mathcal {G}}\subseteq {\mathcal {F}}}$ , the conditional probability ${\displaystyle \operatorname {P} (A\mid {\mathcal {G}})}$  is a version of the conditional expectation of the indicator function for ${\displaystyle A}$ :

${\displaystyle \operatorname {P} (A\mid {\mathcal {G}})=\operatorname {E} (\mathbf {1} _{A}\mid {\mathcal {G}})\;}$ 

An expectation of a random variable with respect to a regular conditional probability is equal to its conditional expectation.

Consider the probability space ${\displaystyle (\Omega ,{\mathcal {F}},\mathbb {P} )}$  and a sub-sigma field ${\displaystyle {\mathcal {A}}\subset {\mathcal {F}}}$ . The sub-sigma field ${\displaystyle {\mathcal {A}}}$  can be loosely interpreted as containing a subset of the information in ${\displaystyle {\mathcal {F}}}$ . For example, we might think of ${\displaystyle \mathbb {P} (B|{\mathcal {A}})}$  as the probability of the event ${\displaystyle B}$  given the information in ${\displaystyle {\mathcal {A}}}$ .

Also recall that an event ${\displaystyle B}$  is independent of a sub-sigma field ${\displaystyle {\mathcal {A}}}$  if ${\displaystyle \mathbb {P} (B|A)=\mathbb {P} (B)}$  for all ${\displaystyle A\in {\mathcal {A}}}$ . It is incorrect to conclude in general that the information in ${\displaystyle {\mathcal {A}}}$  does not tell us anything about the probability of event ${\displaystyle B}$  occurring. This can be shown with a counter-example:

Consider a probability space on the unit interval, ${\displaystyle \Omega =[0,1]}$ . Let ${\displaystyle {\mathcal {G}}}$  be the sigma-field of all countable sets and sets whose complement is countable. So each set in ${\displaystyle {\mathcal {G}}}$  has measure ${\displaystyle 0}$  or ${\displaystyle 1}$  and so is independent of each event in ${\displaystyle {\mathcal {F}}}$ . However, notice that ${\displaystyle {\mathcal {G}}}$  also contains all the singleton events in ${\displaystyle {\mathcal {F}}}$  (those sets which contain only a single ${\displaystyle \omega \in \Omega }$ ). So knowing which of the events in ${\displaystyle {\mathcal {G}}}$  occurred is equivalent to knowing exactly which ${\displaystyle \omega \in \Omega }$  occurred! So in one sense, ${\displaystyle {\mathcal {G}}}$  contains no information about ${\displaystyle {\mathcal {F}}}$  (it is independent of it), and in another sense it contains all the information in ${\displaystyle {\mathcal {F}}}$ .^[5]

• Conditioning (probability)
• Conditional probability
• Regular conditional probability
• Bayes' theorem