Standard probability space

An isomorphism between two probability spaces ${\displaystyle \textstyle (\Omega _{1},{\mathcal {F}}_{1},P_{1})}$, ${\displaystyle \textstyle (\Omega _{2},{\mathcal {F}}_{2},P_{2})}$ is an invertible map ${\displaystyle \textstyle f:\Omega _{1}\to \Omega _{2}}$ such that ${\displaystyle \textstyle f}$ and ${\displaystyle \textstyle f^{-1}}$ both are (measurable and) measure preserving maps.

Two probability spaces are isomorphic if there exists an isomorphism between them.

Two probability spaces ${\displaystyle \textstyle (\Omega _{1},{\mathcal {F}}_{1},P_{1})}$, ${\displaystyle \textstyle (\Omega _{2},{\mathcal {F}}_{2},P_{2})}$ are isomorphic ${\displaystyle \textstyle \operatorname {mod} \,0}$ if there exist null sets ${\displaystyle \textstyle A_{1}\subset \Omega _{1}}$, ${\displaystyle \textstyle A_{2}\subset \Omega _{2}}$ such that the probability spaces ${\displaystyle \textstyle \Omega _{1}\setminus A_{1}}$, ${\displaystyle \textstyle \Omega _{2}\setminus A_{2}}$ are isomorphic (being endowed naturally with sigma-fields and probability measures).

A probability space is standard, if it is isomorphic ${\displaystyle \textstyle \operatorname {mod} \,0}$ to an interval with Lebesgue measure, a finite or countable set of atoms, or a combination (disjoint union) of both.

See (Rokhlin 1952, Sect. 2.4 (p. 20)), (Haezendonck 1973, Proposition 6 (p. 249) and Remark 2 (p. 250)), and (de la Rue 1993, Theorem 4-3). See also (Kechris 1995, Sect. 17.F), and (Itô 1984, especially Sect. 2.4 and Exercise 3.1(v)). In (Petersen 1983, Definition 4.5 on page 16) the measure is assumed finite, not necessarily probabilistic. In (Sinai 1994, Definition 1 on page 16) atoms are not allowed.

The space of all functions ${\displaystyle \textstyle f:\mathbb {R} \to \mathbb {R} }$ may be thought of as the product ${\displaystyle \textstyle \mathbb {R} ^{\mathbb {R} }}$ of a continuum of copies of the real line ${\displaystyle \textstyle \mathbb {R} }$. One may endow ${\displaystyle \textstyle \mathbb {R} }$ with a probability measure, say, the standard normal distribution ${\displaystyle \textstyle \gamma =N(0,1)}$, and treat the space of functions as the product ${\displaystyle \textstyle (\mathbb {R} ,\gamma )^{\mathbb {R} }}$ of a continuum of identical probability spaces ${\displaystyle \textstyle (\mathbb {R} ,\gamma )}$. The product measure ${\displaystyle \textstyle \gamma ^{\mathbb {R} }}$ is a probability measure on ${\displaystyle \textstyle \mathbb {R} ^{\mathbb {R} }}$. Naively it might seem that ${\displaystyle \textstyle \gamma ^{\mathbb {R} }}$ describes white noise.

However, the integral of a white noise function from 0 to 1 should be a random variable distributed N(0, 1). In contrast, the integral (from 0 to 1) of ${\displaystyle \textstyle f\in \textstyle (\mathbb {R} ,\gamma )^{\mathbb {R} }}$ is undefined. ƒ also fails to be almost surely measurable, and the probability of ƒ being measurable is undefined. Indeed, if X is a random variable distributed (say) uniformly on (0, 1) and independent of ƒ, then ƒ(X) is not a random variable at all (it lacks measurability).

Let ${\displaystyle \textstyle Z\subset (0,1)}$ be a set whose inner Lebesgue measure is equal to 0, but outer Lebesgue measure is equal to 1 (thus, ${\displaystyle \textstyle Z}$ is nonmeasurable to extreme). There exists a probability measure ${\displaystyle \textstyle m}$ on ${\displaystyle \textstyle Z}$ such that ${\displaystyle \textstyle m(Z\cap A)=\operatorname {mes} (A)}$ for every Lebesgue measurable ${\displaystyle \textstyle A\subset (0,1)}$. (Here ${\displaystyle \textstyle \operatorname {mes} }$ is the Lebesgue measure.) Events and random variables on the probability space ${\displaystyle \textstyle (Z,m)}$ (treated ${\displaystyle \textstyle \operatorname {mod} \,0}$) are in a natural one-to-one correspondence with events and random variables on the probability space ${\displaystyle \textstyle ((0,1),\operatorname {mes} )}$. It might seem that the probability space ${\displaystyle \textstyle (Z,m)}$ is as good as ${\displaystyle \textstyle ((0,1),\operatorname {mes} )}$.

However, it is not. A random variable ${\displaystyle \textstyle X}$ defined by ${\displaystyle \textstyle X(\omega )=\omega }$ is distributed uniformly on ${\displaystyle \textstyle (0,1)}$. The conditional measure, given ${\displaystyle \textstyle X=x}$, is just a single atom (at ${\displaystyle \textstyle x}$), provided that ${\displaystyle \textstyle ((0,1),\operatorname {mes} )}$ is the underlying probability space. However, if ${\displaystyle \textstyle (Z,m)}$ is used instead, then the conditional measure does not exist when ${\displaystyle \textstyle x\notin Z}$.

A perforated circle is constructed similarly. Its events and random variables are the same as on the usual circle. The group of rotations acts on them naturally. However, it fails to act on the perforated circle.

See also (Rudolph 1990, page 17).

Let ${\displaystyle \textstyle Z\subset (0,1)}$ be as in the previous example. Sets of the form ${\displaystyle \textstyle (A\cap Z)\cup (B\setminus Z),}$ where ${\displaystyle \textstyle A}$ and ${\displaystyle \textstyle B}$ are arbitrary Lebesgue measurable sets, are a σ-algebra ${\displaystyle \textstyle {\mathcal {F}};}$ it contains the Lebesgue σ-algebra and ${\displaystyle \textstyle Z.}$ The formula

${\displaystyle \displaystyle m{\big (}(A\cap Z)\cup (B\setminus Z){\big )}=p\,\operatorname {mes} (A)+(1-p)\operatorname {mes} (B)}$

gives the general form of a probability measure ${\displaystyle \textstyle m}$ on ${\displaystyle \textstyle {\big (}(0,1),{\mathcal {F}}{\big )}}$ that extends the Lebesgue measure; here ${\displaystyle \textstyle p\in [0,1]}$ is a parameter. To be specific, we choose ${\displaystyle \textstyle p=0.5.}$ It might seem that such an extension of the Lebesgue measure is at least harmless.

However, it is the perforated interval in disguise. The map

${\displaystyle f(x)={\begin{cases}0.5x&{\text{for }}x\in Z,\\0.5+0.5x&{\text{for }}x\in (0,1)\setminus Z\end{cases}}}$

is an isomorphism between ${\displaystyle \textstyle {\big (}(0,1),{\mathcal {F}},m{\big )}}$ and the perforated interval corresponding to the set

${\displaystyle \displaystyle Z_{1}=\{0.5x:x\in Z\}\cup \{0.5+0.5x:x\in (0,1)\setminus Z\}\,,}$

another set of inner Lebesgue measure 0 but outer Lebesgue measure 1.

See also (Rudolph 1990, Exercise 2.11 on page 18).

A measurable function ${\displaystyle \textstyle f:\Omega \to \mathbb {R} }$ induces a pushforward measure ${\displaystyle f_{*}P}$, – the probability measure ${\displaystyle \textstyle \mu }$ on ${\displaystyle \textstyle \mathbb {R} ,}$ defined by

${\displaystyle \displaystyle \mu (B)=(f_{*}P)(B)=P{\big (}f^{-1}(B){\big )}}$    for Borel sets ${\displaystyle \textstyle B\subset \mathbb {R} .}$

i.e. the distribution of the random variable ${\displaystyle f}$. The image ${\displaystyle \textstyle f(\Omega )}$ is always a set of full outer measure,

${\displaystyle \displaystyle \mu ^{*}{\big (}f(\Omega ){\big )}=\inf _{B\supset f(\Omega )}\mu (B)=\inf _{B\supset f(\Omega )}P(f^{-1}(B))=P(\Omega )=1,}$

but its inner measure can differ (see a perforated interval). In other words, ${\displaystyle \textstyle f(\Omega )}$ need not be a set of full measure ${\displaystyle \textstyle \mu .}$

A measurable function ${\displaystyle \textstyle f:\Omega \to \mathbb {R} }$ is called generating if ${\displaystyle \textstyle {\mathcal {F}}}$ is the completion with respect to ${\displaystyle P}$ of the σ-algebra of inverse images ${\displaystyle \textstyle f^{-1}(B),}$ where ${\displaystyle \textstyle B\subset \mathbb {R} }$ runs over all Borel sets.

Caution.   The following condition is not sufficient for ${\displaystyle \textstyle f}$ to be generating: for every ${\displaystyle \textstyle A\in {\mathcal {F}}}$ there exists a Borel set ${\displaystyle \textstyle B\subset \mathbb {R} }$ such that ${\displaystyle \textstyle P(A{\mathbin {\Delta }}f^{-1}(B))=0.}$ (${\displaystyle \textstyle \Delta }$ means symmetric difference).

Theorem. Let a measurable function ${\displaystyle \textstyle f:\Omega \to \mathbb {R} }$ be injective and generating, then the following two conditions are equivalent:

• ${\displaystyle \mu (\textstyle f(\Omega ))=1}$ (i.e. the inner measure has also full measure, and the image ${\displaystyle \textstyle f(\Omega )}$ is measureable with respect to the completion);
• ${\displaystyle (\Omega ,{\mathcal {F}},P)\,}$ is a standard probability space.

See also (Itô 1984, Sect. 3.1).

The same theorem holds for any ${\displaystyle \mathbb {R} ^{n}\,}$ (in place of ${\displaystyle \mathbb {R} \,}$). A measurable function ${\displaystyle f:\Omega \to \mathbb {R} ^{n}\,}$ may be thought of as a finite sequence of random variables ${\displaystyle X_{1},\dots ,X_{n}:\Omega \to \mathbb {R} ,\,}$ and ${\displaystyle f\,}$ is generating if and only if ${\displaystyle {\mathcal {F}}\,}$ is the completion of the σ-algebra generated by ${\displaystyle X_{1},\dots ,X_{n}.\,}$

The theorem still holds for the space ${\displaystyle \mathbb {R} ^{\infty }\,}$ of infinite sequences. A measurable function ${\displaystyle f:\Omega \to \mathbb {R} ^{\infty }\,}$ may be thought of as an infinite sequence of random variables ${\displaystyle X_{1},X_{2},\dots$ :\Omega \to \mathbb {R} ,\,} and ${\displaystyle f\,}$ is generating if and only if ${\displaystyle {\mathcal {F}}\,}$ is the completion of the σ-algebra generated by ${\displaystyle X_{1},X_{2},\dots .\,}$

In particular, if the random variables ${\displaystyle X_{n}\,}$ take on only two values 0 and 1, we deal with a measurable function ${\displaystyle f:\Omega \to \{0,1\}^{\infty }\,}$ and a sequence of sets ${\displaystyle A_{1},A_{2},\ldots \in {\mathcal {F}}.\,}$ The function ${\displaystyle f\,}$ is generating if and only if ${\displaystyle {\mathcal {F}}\,}$ is the completion of the σ-algebra generated by ${\displaystyle A_{1},A_{2},\dots .\,}$

In the pioneering work (Rokhlin 1952) sequences ${\displaystyle A_{1},A_{2},\ldots \,}$ that correspond to injective, generating ${\displaystyle f\,}$ are called bases of the probability space ${\displaystyle (\Omega ,{\mathcal {F}},P)\,}$ (see Rokhlin 1952, Sect. 2.1). A basis is called complete mod 0, if ${\displaystyle f(\Omega )\,}$ is of full measure ${\displaystyle \mu ,\,}$ see (Rokhlin 1952, Sect. 2.2). In the same section Rokhlin proved that if a probability space is complete mod 0 with respect to some basis, then it is complete mod 0 with respect to every other basis, and defines Lebesgue spaces by this completeness property. See also (Haezendonck 1973, Prop. 4 and Def. 7) and (Rudolph 1990, Sect. 2.3, especially Theorem 2.2).

The four cases treated above are mutually equivalent, and can be united, since the measurable spaces ${\displaystyle \mathbb {R} ,\,}$ ${\displaystyle \mathbb {R} ^{n},\,}$ ${\displaystyle \mathbb {R} ^{\infty }\,}$ and ${\displaystyle \{0,1\}^{\infty }\,}$ are mutually isomorphic; they all are standard measurable spaces (in other words, standard Borel spaces).

Existence of an injective measurable function from ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$ to a standard measurable space ${\displaystyle \textstyle (X,\Sigma )}$ does not depend on the choice of ${\displaystyle \textstyle (X,\Sigma ).}$ Taking ${\displaystyle \textstyle (X,\Sigma )=\{0,1\}^{\infty }}$ we get the property well known as being countably separated (but called separable in Itô 1984).

Existence of a generating measurable function from ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$ to a standard measurable space ${\displaystyle \textstyle (X,\Sigma )}$ also does not depend on the choice of ${\displaystyle \textstyle (X,\Sigma ).}$ Taking ${\displaystyle \textstyle (X,\Sigma )=\{0,1\}^{\infty }}$ we get the property well known as being countably generated (mod 0), see (Durrett 1996, Exer. I.5).

Every injective measurable function from a standard probability space to a standard measurable space is generating. See (Rokhlin 1952, Sect. 2.5), (Haezendonck 1973, Corollary 2 on page 253), (de la Rue 1993, Theorems 3-4 and 3-5). This property does not hold for the non-standard probability space dealt with in the subsection "A superfluous measurable set" above.

Caution.   The property of being countably generated is invariant under mod 0 isomorphisms, but the property of being countably separated is not. In fact, a standard probability space ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$ is countably separated if and only if the cardinality of ${\displaystyle \textstyle \Omega }$ does not exceed continuum (see Itô 1984, Exer. 3.1(v)). A standard probability space may contain a null set of any cardinality, thus, it need not be countably separated. However, it always contains a countably separated subset of full measure.

Definition.   ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$ is standard if it is countably separated, countably generated, and absolutely measurable.

See (Rokhlin 1952, the end of Sect. 2.3) and (Haezendonck 1973, Remark 2 on page 248). "Absolutely measurable" means: measurable in every countably separated, countably generated probability space containing it.

Definition.   ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$ is standard if it is countably separated and perfect.

See (Itô 1984, Sect. 3.1). "Perfect" means that for every measurable function from ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$ to ${\displaystyle \mathbb {R} \,}$ the image measure is regular. (Here the image measure is defined on all sets whose inverse images belong to ${\displaystyle \textstyle {\mathcal {F}}}$, irrespective of the Borel structure of ${\displaystyle \mathbb {R} \,}$).

Definition.   ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$ is standard if there exists a topology ${\displaystyle \textstyle \tau }$ on ${\displaystyle \textstyle \Omega }$ such that

• the topological space ${\displaystyle \textstyle (\Omega ,\tau )}$ is metrizable;
• ${\displaystyle \textstyle {\mathcal {F}}}$ is the completion of the σ-algebra generated by ${\displaystyle \textstyle \tau }$ (that is, by all open sets);
• for every ${\displaystyle \textstyle \varepsilon >0}$ there exists a compact set ${\displaystyle \textstyle K}$ in ${\displaystyle \textstyle (\Omega ,\tau )}$ such that ${\displaystyle \textstyle P(K)\geq 1-\varepsilon .}$

See (de la Rue 1993, Sect. 1).

In the discrete setup, the conditional probability is another probability measure, and the conditional expectation may be treated as the (usual) expectation with respect to the conditional measure, see conditional expectation. In the non-discrete setup, conditioning is often treated indirectly, since the condition may have probability 0, see conditional expectation. As a result, a number of well-known facts have special 'conditional' counterparts. For example: linearity of the expectation; Jensen's inequality (see conditional expectation); Hölder's inequality; the monotone convergence theorem, etc.

Given a random variable ${\displaystyle \textstyle Y}$ on a probability space ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$, it is natural to try constructing a conditional measure ${\displaystyle \textstyle P_{y}}$, that is, the conditional distribution of ${\displaystyle \textstyle \omega \in \Omega }$ given ${\displaystyle \textstyle Y(\omega )=y}$. In general this is impossible (see Durrett 1996, Sect. 4.1(c)). However, for a standard probability space ${\displaystyle \textstyle (\Omega ,{\mathcal {F}},P)}$ this is possible, and well known as canonical system of measures (see Rokhlin 1952, Sect. 3.1), which is basically the same as conditional probability measures (see Itô 1984, Sect. 3.5), disintegration of measure (see Kechris 1995, Exercise (17.35)), and regular conditional probabilities (see Durrett 1996, Sect. 4.1(c)).

The conditional Jensen's inequality is just the (usual) Jensen's inequality applied to the conditional measure. The same holds for many other facts.

Given two probability spaces ${\displaystyle \textstyle (\Omega _{1},{\mathcal {F}}_{1},P_{1})}$, ${\displaystyle \textstyle (\Omega _{2},{\mathcal {F}}_{2},P_{2})}$ and a measure preserving map ${\displaystyle \textstyle f:\Omega _{1}\to \Omega _{2}}$, the image ${\displaystyle \textstyle f(\Omega _{1})}$ need not cover the whole ${\displaystyle \textstyle \Omega _{2}}$, it may miss a null set. It may seem that ${\displaystyle \textstyle P_{2}(f(\Omega _{1}))}$ has to be equal to 1, but it is not so. The outer measure of ${\displaystyle \textstyle f(\Omega _{1})}$ is equal to 1, but the inner measure may differ. However, if the probability spaces ${\displaystyle \textstyle (\Omega _{1},{\mathcal {F}}_{1},P_{1})}$, ${\displaystyle \textstyle (\Omega _{2},{\mathcal {F}}_{2},P_{2})}$ are standard then ${\displaystyle \textstyle P_{2}(f(\Omega _{1}))=1}$, see (de la Rue 1993, Theorem 3-2). If ${\displaystyle \textstyle f}$ is also one-to-one then every ${\displaystyle \textstyle A\in {\mathcal {F}}_{1}}$ satisfies ${\displaystyle \textstyle f(A)\in {\mathcal {F}}_{2}}$, ${\displaystyle \textstyle P_{2}(f(A))=P_{1}(A)}$. Therefore, ${\displaystyle \textstyle f^{-1}}$ is measurable (and measure preserving). See (Rokhlin 1952, Sect. 2.5 (p. 20)) and (de la Rue 1993, Theorem 3-5). See also (Haezendonck 1973, Proposition 9 (and Remark after it)).

"There is a coherent way to ignore the sets of measure 0 in a measure space" (Petersen 1983, page 15). Striving to get rid of null sets, mathematicians often use equivalence classes of measurable sets or functions. Equivalence classes of measurable subsets of a probability space form a normed complete Boolean algebra called the measure algebra (or metric structure). Every measure preserving map ${\displaystyle \textstyle f:\Omega _{1}\to \Omega _{2}}$ leads to a homomorphism ${\displaystyle \textstyle F}$ of measure algebras; basically, ${\displaystyle \textstyle F(B)=f^{-1}(B)}$ for ${\displaystyle \textstyle B\in {\mathcal {F}}_{2}}$.

It may seem that every homomorphism of measure algebras has to correspond to some measure preserving map, but it is not so. However, for standard probability spaces each ${\displaystyle \textstyle F}$ corresponds to some ${\displaystyle \textstyle f}$. See (Rokhlin 1952, Sect. 2.6 (p. 23) and 3.2), (Kechris 1995, Sect. 17.F), (Petersen 1983, Theorem 4.7 on page 17).