Doob's martingale convergence theorems

The proof is simplified by making the (stronger) assumption that the supermartingale is uniformly bounded; that is, there is a constant ${\displaystyle M}$ such that ${\displaystyle |X_{n}|\leq M}$ always holds. In the event that the sequence ${\displaystyle X_{1},X_{2},\dots }$ does not converge, then ${\displaystyle \liminf X_{n}}$ and ${\displaystyle \limsup X_{n}}$ differ. If also the sequence is bounded, then there are some real numbers ${\displaystyle a}$ and ${\displaystyle b}$ such that ${\displaystyle a<b}$ and the sequence crosses the interval ${\displaystyle [a,b]}$ infinitely often. That is, the sequence is eventually less than ${\displaystyle a}$, and at a later time exceeds ${\displaystyle b}$, and at an even later time is less than ${\displaystyle a}$, and so forth ad infinitum. These periods where the sequence starts below ${\displaystyle a}$ and later exceeds ${\displaystyle b}$ are called "upcrossings".

Consider a stock market game in which at time ${\displaystyle t}$, one may buy or sell shares of the stock at price ${\displaystyle X_{t}}$. On the one hand, it can be shown from the definition of a supermartingale that for any ${\displaystyle N\in \mathbf {N} }$ there is no strategy which maintains a non-negative amount of stock and has positive expected profit after playing this game for ${\displaystyle N}$ steps. On the other hand, if the prices cross a fixed interval ${\displaystyle [a,b]}$ very often, then the following strategy seems to do well: buy the stock when the price drops below ${\displaystyle a}$, and sell it when the price exceeds ${\displaystyle b}$. Indeed, if ${\displaystyle u_{N}}$ is the number of upcrossings in the sequence by time ${\displaystyle N}$, then the profit at time ${\displaystyle N}$ is at least ${\displaystyle (b-a)u_{N}-2M}$: each upcrossing provides at least ${\displaystyle b-a}$ profit, and if the last action was a "buy", then in the worst case the buying price was ${\displaystyle a\leq M}$ and the current price is ${\displaystyle -M}$. But any strategy has expected profit at most ${\displaystyle 0}$, so necessarily

${\displaystyle \operatorname {E} {\big [}u_{N}{\big ]}\leq {\frac {2M}{b-a}}.}$

By the monotone convergence theorem for expectations, this means that

${\displaystyle \operatorname {E} {\big [}\lim _{N\to \infty }u_{N}{\big ]}\leq {\frac {2M}{b-a}},}$

so the expected number of upcrossings in the whole sequence is finite. It follows that the infinite-crossing event for interval ${\displaystyle [a,b]}$ occurs with probability ${\displaystyle 0}$. By a union bound over all rational ${\displaystyle a}$ and ${\displaystyle b}$, with probability ${\displaystyle 1}$, no interval exists which is crossed infinitely often. If for all ${\displaystyle a,b\in \mathbf {Q} }$ there are finitely many upcrossings of interval ${\displaystyle [a,b]}$, then the limit inferior and limit superior of the sequence must agree, so the sequence must converge. This shows that the martingale converges with probability ${\displaystyle 1}$.

Under the conditions of the martingale convergence theorem given above, it is not necessarily true that the supermartingale ${\displaystyle (X_{n})_{n\in \mathbf {N} }}$ converges in mean (i.e. that ${\displaystyle \lim _{n\to \infty }\operatorname {E} [|X_{n}-X|]=0}$).

As an example,[2] let ${\displaystyle (X_{n})_{n\in \mathbf {N} }}$ be a ${\displaystyle \pm 1}$ random walk with ${\displaystyle X_{0}=1}$. Let ${\displaystyle N}$ be the first time when ${\displaystyle X_{n}=0}$, and let ${\displaystyle (Y_{n})_{n\in \mathbf {N} }}$ be the stochastic process defined by ${\displaystyle Y_{n}:=X_{\min(N,n)}}$. Then ${\displaystyle N}$ is a stopping time with respect to the martingale ${\displaystyle (X_{n})_{n\in \mathbf {N} }}$, so ${\displaystyle (Y_{n})_{n\in \mathbf {N} }}$ is also a martingale, referred to as a stopped martingale. In particular, ${\displaystyle (Y_{n})_{n\in \mathbf {N} }}$ is a supermartingale which is bounded below, so by the martingale convergence theorem it converges pointwise almost surely to a random variable ${\displaystyle Y}$. But if ${\displaystyle Y_{n}>0}$ then ${\displaystyle Y_{n+1}=Y_{n}\pm 1}$, so ${\displaystyle Y}$ is almost surely zero.

This means that ${\displaystyle \operatorname {E} [Y]=0}$. However, ${\displaystyle \operatorname {E} [Y_{n}]=1}$ for every ${\displaystyle n\geq 1}$, since ${\displaystyle (Y_{n})_{n\in \mathbf {N} }}$ is a random walk which starts at ${\displaystyle 1}$ and subsequently makes mean-zero moves (alternately, note that ${\displaystyle \operatorname {E} [Y_{n}]=\operatorname {E} [Y_{0}]=1}$ since ${\displaystyle (Y_{n})_{n\in \mathbf {N} }}$ is a martingale). Therefore ${\displaystyle (Y_{n})_{n\in \mathbf {N} }}$ cannot converge to ${\displaystyle Y}$ in mean. Moreover, if ${\displaystyle (Y_{n})_{n\in \mathbb {N} }}$ were to converge in mean to any random variable ${\displaystyle R}$, then some subsequence converges to ${\displaystyle R}$ almost surely. So by the above argument ${\displaystyle R=0}$ almost surely, which contradicts convergence in mean.

Doob's first martingale convergence theorem provides a sufficient condition for the random variables ${\displaystyle N_{t}}$ to have a limit as ${\displaystyle t\to +\infty }$ in a pointwise sense, i.e. for each ${\displaystyle \omega }$ in the sample space ${\displaystyle \Omega }$ individually.

For ${\displaystyle t\geq 0}$, let ${\displaystyle N_{t}^{-}=\max(-N_{t},0)}$ and suppose that

${\displaystyle \sup _{t>0}\operatorname {E} {\big [}N_{t}^{-}{\big ]}<+\infty .}$

Then the pointwise limit

${\displaystyle N(\omega )=\lim _{t\to +\infty }N_{t}(\omega )}$

exists and is finite for ${\displaystyle \mathbf {P} }$-almost all ${\displaystyle \omega \in \Omega }$.[3]

It is important to note that the convergence in Doob's first martingale convergence theorem is pointwise, not uniform, and is unrelated to convergence in mean square, or indeed in any L^p space. In order to obtain convergence in L¹ (i.e., convergence in mean), one requires uniform integrability of the random variables ${\displaystyle N_{t}}$. By Chebyshev's inequality, convergence in L¹ implies convergence in probability and convergence in distribution.

The following are equivalent:

• ${\displaystyle (N_{t})_{t>0}}$ is uniformly integrable, i.e.

${\displaystyle \lim _{C\to \infty }\sup _{t>0}\int _{\{\omega \in \Omega \,\mid \,|N_{t}(\omega )|>C\}}\left|N_{t}(\omega )\right|\,\mathrm {d} \mathbf {P} (\omega )=0;}$

• there exists an integrable random variable ${\displaystyle N\in L^{1}(\Omega ,\mathbf {P}$ ;\mathbf {R} )} such that ${\displaystyle N_{t}\to N}$ as ${\displaystyle t\to \infty }$ both ${\displaystyle \mathbf {P} }$-almost surely and in ${\displaystyle L^{1}(\Omega ,\mathbf {P}$ ;\mathbf {R} )} , i.e.

${\displaystyle \operatorname {E} \left[\left|N_{t}-N\right|\right]=\int _{\Omega }\left|N_{t}(\omega )-N(\omega )\right|\,\mathrm {d} \mathbf {P} (\omega )\to 0{\text{ as }}t\to +\infty .}$

Let ${\displaystyle M:[0,\infty )\times \Omega \to \mathbf {R} }$ be a continuous martingale such that

${\displaystyle \sup _{t>0}\operatorname {E} {\big [}{\big |}M_{t}{\big |}^{p}{\big ]}<+\infty }$

for some ${\displaystyle p>1}$. Then there exists a random variable ${\displaystyle M\in L^{p}(\Omega ,\mathbf {P}$ ;\mathbf {R} )} such that ${\displaystyle M_{t}\to M}$ as ${\displaystyle t\to +\infty }$ both ${\displaystyle \mathbf {P} }$-almost surely and in ${\displaystyle L^{p}(\Omega ,\mathbf {P}$ ;\mathbf {R} )} .

The statement for discrete-time martingales is essentially identical, with the obvious difference that the continuity assumption is no longer necessary.

Doob's martingale convergence theorems imply that conditional expectations also have a convergence property.

Let ${\displaystyle (\Omega ,F,\mathbf {P} )}$ be a probability space and let ${\displaystyle X}$ be a random variable in ${\displaystyle L^{1}}$. Let ${\displaystyle F_{*}=(F_{k})_{k\in \mathbf {N} }}$ be any filtration of ${\displaystyle F}$, and define ${\displaystyle F_{\infty }}$ to be the minimal σ-algebra generated by ${\displaystyle (F_{k})_{k\in \mathbf {N} }}$. Then

${\displaystyle \operatorname {E} {\big [}X\mid F_{k}{\big ]}\to \operatorname {E} {\big [}X\mid F_{\infty }{\big ]}{\text{ as }}k\to \infty }$

both ${\displaystyle \mathbf {P} }$-almost surely and in ${\displaystyle L^{1}}$.

This result is usually called Lévy's zero–one law or Levy's upwards theorem. The reason for the name is that if ${\displaystyle A}$ is an event in ${\displaystyle F_{\infty }}$, then the theorem says that ${\displaystyle \mathbf {P} [A\mid F_{k}]\to \mathbf {1} _{A}}$ almost surely, i.e., the limit of the probabilities is 0 or 1. In plain language, if we are learning gradually all the information that determines the outcome of an event, then we will become gradually certain what the outcome will be. This sounds almost like a tautology, but the result is still non-trivial. For instance, it easily implies Kolmogorov's zero–one law, since it says that for any tail event A, we must have ${\displaystyle \mathbf {P} [A]=\mathbf {1} _{A}}$ almost surely, hence ${\displaystyle \mathbf {P} [A]\in \{0,1\}}$.

Similarly we have the Levy's downwards theorem :

Let ${\displaystyle (\Omega ,F,\mathbf {P} )}$ be a probability space and let ${\displaystyle X}$ be a random variable in ${\displaystyle L^{1}}$. Let ${\displaystyle (F_{k})_{k\in \mathbf {N} }}$ be any decreasing sequence of sub-sigma algebras of ${\displaystyle F}$, and define ${\displaystyle F_{\infty }}$ to be the intersection. Then

${\displaystyle \operatorname {E} {\big [}X\mid F_{k}{\big ]}\to \operatorname {E} {\big [}X\mid F_{\infty }{\big ]}{\text{ as }}k\to \infty }$

both ${\displaystyle \mathbf {P} }$-almost surely and in ${\displaystyle L^{1}}$.