Conditional probability distribution

For discrete random variables, the conditional probability mass function of ${\displaystyle Y}$ given ${\displaystyle X=x}$ can be written according to its definition as:

Due to the occurrence of ${\displaystyle P(X=x)}$ in the denominator, this is defined only for non-zero (hence strictly positive) ${\displaystyle P(X=x).}$

The relation with the probability distribution of ${\displaystyle X}$ given ${\displaystyle Y}$ is:

${\displaystyle P(Y=y\mid X=x)P(X=x)=P(\{X=x\}\cap \{Y=y\})=P(X=x\mid Y=y)P(Y=y).}$

Similarly for continuous random variables, the conditional probability density function of ${\displaystyle Y}$ given the occurrence of the value ${\displaystyle x}$ of ${\displaystyle X}$ can be written as[2]^{: p. 99}

where ${\displaystyle f_{X,Y}(x,y)}$ gives the joint density of ${\displaystyle X}$ and ${\displaystyle Y}$, while ${\displaystyle f_{X}(x)}$ gives the marginal density for ${\displaystyle X}$. Also in this case it is necessary that ${\displaystyle f_{X}(x)>0}$.

The relation with the probability distribution of ${\displaystyle X}$ given ${\displaystyle Y}$ is given by:

${\displaystyle f_{Y\mid X}(y\mid x)f_{X}(x)=f_{X,Y}(x,y)=f_{X|Y}(x\mid y)f_{Y}(y).}$

The concept of the conditional distribution of a continuous random variable is not as intuitive as it might seem: Borel's paradox shows that conditional probability density functions need not be invariant under coordinate transformations.

Example

Bivariate normal joint density

The graph shows a bivariate normal joint density for random variables ${\displaystyle X}$ and ${\displaystyle Y}$. To see the distribution of ${\displaystyle Y}$ conditional on ${\displaystyle X=70}$, one can first visualize the line ${\displaystyle X=70}$ in the ${\displaystyle X,Y}$ plane, and then visualize the plane containing that line and perpendicular to the ${\displaystyle X,Y}$ plane. The intersection of that plane with the joint normal density, once rescaled to give unit area under the intersection, is the relevant conditional density of ${\displaystyle Y}$.

${\displaystyle Y\mid X=70\ \sim \ {\mathcal {N}}\left(\mu _{Y}+{\frac {\sigma _{Y}}{\sigma _{X}}}\rho (70-\mu _{X}),\,(1-\rho ^{2})\sigma _{Y}^{2}\right).}$

Random variables ${\displaystyle X}$, ${\displaystyle Y}$ are independent if and only if the conditional distribution of ${\displaystyle Y}$ given ${\displaystyle X}$ is, for all possible realizations of ${\displaystyle X}$, equal to the unconditional distribution of ${\displaystyle Y}$. For discrete random variables this means ${\displaystyle P(Y=y|X=x)=P(Y=y)}$ for all possible ${\displaystyle y}$ and ${\displaystyle x}$ with ${\displaystyle P(X=x)>0}$. For continuous random variables ${\displaystyle X}$ and ${\displaystyle Y}$, having a joint density function, it means ${\displaystyle f_{Y}(y|X=x)=f_{Y}(y)}$ for all possible ${\displaystyle y}$ and ${\displaystyle x}$ with ${\displaystyle f_{X}(x)>0}$.

Seen as a function of ${\displaystyle y}$ for given ${\displaystyle x}$, ${\displaystyle P(Y=y|X=x)}$ is a probability mass function and so the sum over all ${\displaystyle y}$ (or integral if it is a conditional probability density) is 1. Seen as a function of ${\displaystyle x}$ for given ${\displaystyle y}$, it is a likelihood function, so that the sum over all ${\displaystyle x}$ need not be 1.

Additionally, a marginal of a joint distribution can be expressed as the expectation of the corresponding conditional distribution. For instance, ${\displaystyle p_{X}(x)=E_{Y}[p_{X|Y}(X\ |\ Y)]}$.

Let ${\displaystyle (\Omega ,{\mathcal {F}},P)}$ be a probability space, ${\displaystyle {\mathcal {G}}\subseteq {\mathcal {F}}}$ a ${\displaystyle \sigma }$-field in ${\displaystyle {\mathcal {F}}}$. Given ${\displaystyle A\in {\mathcal {F}}}$, the Radon-Nikodym theorem implies that there is[3] a ${\displaystyle {\mathcal {G}}}$-measurable random variable ${\displaystyle P(A\mid {\mathcal {G}}):\Omega \to \mathbb {R} }$, called the conditional probability, such that

$${\displaystyle \int _{G}P(A\mid {\mathcal {G}})(\omega )dP(\omega )=P(A\cap G)}$$

for every ${\displaystyle G\in {\mathcal {G}}}$, and such a random variable is uniquely defined up to sets of probability zero. A conditional probability is called regular if ${\displaystyle \operatorname {P} (\cdot \mid {\mathcal {G}})(\omega )}$ is a probability measure on ${\displaystyle (\Omega ,{\mathcal {F}})}$ for all ${\displaystyle \omega \in \Omega }$ a.e.

Special cases:

• For the trivial sigma algebra ${\displaystyle {\mathcal {G}}=\{\emptyset ,\Omega \}}$, the conditional probability is the constant function ${\displaystyle \operatorname {P} \!\left(A\mid \{\emptyset ,\Omega \}\right)=\operatorname {P} (A).}$
• If ${\displaystyle A\in {\mathcal {G}}}$, then ${\displaystyle \operatorname {P} (A\mid {\mathcal {G}})=1_{A}}$, the indicator function (defined below).

Let ${\displaystyle X:\Omega \to E}$ be a ${\displaystyle (E,{\mathcal {E}})}$-valued random variable. For each ${\displaystyle B\in {\mathcal {E}}}$, define

$${\displaystyle \mu _{X\,|\,{\mathcal {G}}}(B\,|\,{\mathcal {G}})=\mathrm {P} (X^{-1}(B)\,|\,{\mathcal {G}}).}$$

For any ${\displaystyle \omega \in \Omega }$, the function ${\displaystyle \mu _{X\,|{\mathcal {G}}}(\cdot \,|{\mathcal {G}})(\omega ):{\mathcal {E}}\to \mathbb {R} }$ is called the conditional probability distribution of ${\displaystyle X}$ given ${\displaystyle {\mathcal {G}}}$. If it is a probability measure on ${\displaystyle (E,{\mathcal {E}})}$, then it is called regular.

For a real-valued random variable (with respect to the Borel ${\displaystyle \sigma }$-field ${\displaystyle {\mathcal {R}}^{1}}$ on ${\displaystyle \mathbb {R} }$), every conditional probability distribution is regular.[4] In this case,${\displaystyle E[X\mid {\mathcal {G}}]=\int _{-\infty }^{\infty }x\,\mu (dx,\cdot )}$ almost surely.

• Conditioning (probability)
• Conditional probability
• Regular conditional probability
• Bayes' theorem