Wald's equation

Let N be an integrable, ${\displaystyle \mathbb {N} }$₀-valued random variable, which is independent of the integrable, real-valued random variable Z with E[Z] = 0. Define X_n = (–1)ⁿ Z for all n ∈ ${\displaystyle \mathbb {N} }$. Then assumptions (1), (5), (7), and (8) with C := E[|Z|] are satisfied, hence also (2) and (6), and Wald's equation applies. If the distribution of Z is not symmetric, then (9) does not hold. Note that, when Z is not almost surely equal to the zero random variable, then (11) and (12) cannot hold simultaneously for any filtration (F_n)_{n∈${\displaystyle \mathbb {N} }$}, because Z cannot be independent of itself as E[Z²] = (E[Z])² = 0 is impossible.

Let (X_n)_{n∈${\displaystyle \mathbb {N} }$} be a sequence of independent, symmetric, and {–1, +1}-valued random variables. For every n ∈ ${\displaystyle \mathbb {N} }$ let F_n be the σ-algebra generated by X₁, . . . , X_n and define N = n when X_n is the first random variable taking the value +1. Note that P(N = n) = 1/2ⁿ, hence E[N] < ∞ by the ratio test. The assumptions (1), (5) and (9), hence (4) and (8) with C = 1, (10), (11), and (12) hold, hence also (2), and (6) and Wald's equation applies. However, (7) does not hold, because N is defined in terms of the sequence (X_n)_{n∈${\displaystyle \mathbb {N} }$}. Intuitively, one might expect to have E[S_N] > 0 in this example, because the summation stops right after a one, thereby apparently creating a positive bias. However, Wald's equation shows that this intuition is misleading.

Consider a sequence (X_n)_{n∈${\displaystyle \mathbb {N} }$} of i.i.d. (Independent and identically distributed random variables) random variables, taking each of the two values 0 and 1 with probability 1/2 (actually, only X₁ is needed in the following). Define N = 1 – X₁. Then S_N is identically equal to zero, hence E[S_N] = 0, but E[X₁] = 1/2 and E[N] = 1/2 and therefore Wald's equation does not hold. Indeed, the assumptions (1), (3), (4) and (5) are satisfied, however, the equation in assumption (2) holds for all n ∈ ${\displaystyle \mathbb {N} }$ except for n = 1.

Very similar to the second example above, let (X_n)_{n∈${\displaystyle \mathbb {N} }$} be a sequence of independent, symmetric random variables, where X_n takes each of the values 2ⁿ and –2ⁿ with probability 1/2. Let N be the first n ∈ ${\displaystyle \mathbb {N} }$ such that X_n = 2ⁿ. Then, as above, N has finite expectation, hence assumption (5) holds. Since E[X_n] = 0 for all n ∈ ${\displaystyle \mathbb {N} }$, assumptions (1) and (4) hold. However, since S_N = 1 almost surely, Wald's equation cannot hold.

Since N is a stopping time with respect to the filtration generated by (X_n)_{n∈${\displaystyle \mathbb {N} }$}, assumption (2) holds, see above. Therefore, only assumption (3) can fail, and indeed, since

${\displaystyle \{N\geq n\}=\{X_{i}=-2^{i}{\text{ for }}i=1,\ldots ,n-1\}}$

and therefore P(N ≥ n) = 1/2^n–1 for every n ∈ ${\displaystyle \mathbb {N} }$, it follows that

${\displaystyle \sum _{n=1}^{\infty }\operatorname {E} \!{\bigl [}|X_{n}|1_{\{N\geq n\}}{\bigr ]}=\sum _{n=1}^{\infty }2^{n}\,\operatorname {P} (N\geq n)=\sum _{n=1}^{\infty }2=\infty .}$

We first show that the random sum S_N is integrable. Define the partial sums

${\displaystyle S_{i}=\sum _{n=1}^{i}X_{n},\quad i\in {\mathbb {N} }_{0}.}$

(14)

Since N takes its values in ${\displaystyle \mathbb {N} }$₀ and since S₀ = 0, it follows that

${\displaystyle |S_{N}|=\sum _{i=1}^{\infty }|S_{i}|\,1_{\{N=i\}}.}$

The Lebesgue monotone convergence theorem implies that

${\displaystyle \operatorname {E} [|S_{N}|]=\sum _{i=1}^{\infty }\operatorname {E} [|S_{i}|\,1_{\{N=i\}}].}$

By the triangle inequality,

${\displaystyle |S_{i}|\leq \sum _{n=1}^{i}|X_{n}|,\quad i\in {\mathbb {N} }.}$

Using this upper estimate and changing the order of summation (which is permitted because all terms are non-negative), we obtain

${\displaystyle \operatorname {E} [|S_{N}|]\leq \sum _{n=1}^{\infty }\sum _{i=n}^{\infty }\operatorname {E} [|X_{n}|\,1_{\{N=i\}}]=\sum _{n=1}^{\infty }\operatorname {E} [|X_{n}|\,1_{\{N\geq n\}}],}$

(15)

where the second inequality follows using the monotone convergence theorem. By assumption (3), the infinite sequence on the right-hand side of (15) converges, hence S_N is integrable.

We now show that the random sum T_N is integrable. Define the partial sums

${\displaystyle T_{i}=\sum _{n=1}^{i}\operatorname {E} [X_{n}],\quad i\in {\mathbb {N} }_{0},}$

(16)

of real numbers. Since N takes its values in ${\displaystyle \mathbb {N} }$₀ and since T₀ = 0, it follows that

${\displaystyle |T_{N}|=\sum _{i=1}^{\infty }|T_{i}|\,1_{\{N=i\}}.}$

As in step 1, the Lebesgue monotone convergence theorem implies that

${\displaystyle \operatorname {E} [|T_{N}|]=\sum _{i=1}^{\infty }|T_{i}|\operatorname {P} (N=i).}$

By the triangle inequality,

${\displaystyle |T_{i}|\leq \sum _{n=1}^{i}{\bigl |}\!\operatorname {E} [X_{n}]{\bigr |},\quad i\in {\mathbb {N} }.}$

Using this upper estimate and changing the order of summation (which is permitted because all terms are non-negative), we obtain

${\displaystyle \operatorname {E} [|T_{N}|]\leq \sum _{n=1}^{\infty }{\bigl |}\!\operatorname {E} [X_{n}]{\bigr |}\underbrace {\sum _{i=n}^{\infty }\operatorname {P} (N=i)} _{=\,\operatorname {P} (N\geq n)},}$

(17)

By assumption (2),

${\displaystyle {\bigl |}\!\operatorname {E} [X_{n}]{\bigr |}\operatorname {P} (N\geq n)={\bigl |}\!\operatorname {E} [X_{n}1_{\{N\geq n\}}]{\bigr |}\leq \operatorname {E} [|X_{n}|1_{\{N\geq n\}}],\quad n\in {\mathbb {N} }.}$

Substituting this into (17) yields

${\displaystyle \operatorname {E} [|T_{N}|]\leq \sum _{n=1}^{\infty }\operatorname {E} [|X_{n}|1_{\{N\geq n\}}],}$

which is finite by assumption (3), hence T_N is integrable.

To prove Wald's equation, we essentially go through the same steps again without the absolute value, making use of the integrability of the random sums S_N and T_N in order to show that they have the same expectation.

Using the dominated convergence theorem with dominating random variable |S_N| and the definition of the partial sum S_i given in (14), it follows that

${\displaystyle \operatorname {E} [S_{N}]=\sum _{i=1}^{\infty }\operatorname {E} [S_{i}1_{\{N=i\}}]=\sum _{i=1}^{\infty }\sum _{n=1}^{i}\operatorname {E} [X_{n}1_{\{N=i\}}].}$

Due to the absolute convergence proved in (15) above using assumption (3), we may rearrange the summation and obtain that

${\displaystyle \operatorname {E} [S_{N}]=\sum _{n=1}^{\infty }\sum _{i=n}^{\infty }\operatorname {E} [X_{n}1_{\{N=i\}}]=\sum _{n=1}^{\infty }\operatorname {E} [X_{n}1_{\{N\geq n\}}],}$

where we used assumption (1) and the dominated convergence theorem with dominating random variable |X_n| for the second equality. Due to assumption (2) and the σ-additivity of the probability measure,

${\displaystyle {\begin{aligned}\operatorname {E} [X_{n}1_{\{N\geq n\}}]&=\operatorname {E} [X_{n}]\operatorname {P} (N\geq n)\\&=\operatorname {E} [X_{n}]\sum _{i=n}^{\infty }\operatorname {P} (N=i)=\sum _{i=n}^{\infty }\operatorname {E} \!{\bigl [}\operatorname {E} [X_{n}]1_{\{N=i\}}{\bigr ]}.\end{aligned}}}$

Substituting this result into the previous equation, rearranging the summation (which is permitted due to absolute convergence, see (15) above), using linearity of expectation and the definition of the partial sum T_i of expectations given in (16),

${\displaystyle \operatorname {E} [S_{N}]=\sum _{i=1}^{\infty }\sum _{n=1}^{i}\operatorname {E} \!{\bigl [}\operatorname {E} [X_{n}]1_{\{N=i\}}{\bigr ]}=\sum _{i=1}^{\infty }\operatorname {E} [\underbrace {T_{i}1_{\{N=i\}}} _{=\,T_{N}1_{\{N=i\}}}].}$

By using dominated convergence again with dominating random variable |T_N|,

${\displaystyle \operatorname {E} [S_{N}]=\operatorname {E} \!{\biggl [}T_{N}\underbrace {\sum _{i=1}^{\infty }1_{\{N=i\}}} _{=\,1_{\{N\geq 1\}}}{\biggr ]}=\operatorname {E} [T_{N}].}$

If assumptions (4) and (5) are satisfied, then by linearity of expectation,

${\displaystyle \operatorname {E} [T_{N}]=\operatorname {E} \!{\biggl [}\sum _{n=1}^{N}\operatorname {E} [X_{n}]{\biggr ]}=\operatorname {E} [X_{1}]\operatorname {E} \!{\biggl [}\underbrace {\sum _{n=1}^{N}1} _{=\,N}{\biggr ]}=\operatorname {E} [N]\operatorname {E} [X_{1}].}$

This completes the proof.