Cumulant

Some writers[2][3] prefer to define the cumulant-generating function as the natural logarithm of the characteristic function, which is sometimes also called the second characteristic function,[4][5]

${\displaystyle H(t)=\log \operatorname {E} \left[e^{itX}\right]=\sum _{n=1}^{\infty }\kappa _{n}{\frac {(it)^{n}}{n!}}=\mu it-\sigma ^{2}{\frac {t^{2}}{2}}+\cdots }$

An advantage of H(t)—in some sense the function K(t) evaluated for purely imaginary arguments—is that E[e^itX] is well defined for all real values of t even when E[e^tX] is not well defined for all real values of t, such as can occur when there is "too much" probability that X has a large magnitude. Although the function H(t) will be well defined, it will nonetheless mimic K(t) in terms of the length of its Maclaurin series, which may not extend beyond (or, rarely, even to) linear order in the argument t, and in particular the number of cumulants that are well defined will not change. Nevertheless, even when H(t) does not have a long Maclaurin series, it can be used directly in analyzing and, particularly, adding random variables. Both the Cauchy distribution (also called the Lorentzian) and more generally, stable distributions (related to the Lévy distribution) are examples of distributions for which the power-series expansions of the generating functions have only finitely many well-defined terms.

All of the higher cumulants are polynomial functions of the central moments, with integer coefficients, but only in degrees 2 and 3 are the cumulants actually central moments.

• ${\textstyle \kappa _{1}(X)=\operatorname {E} (X)={}}$mean
• ${\textstyle \kappa _{2}(X)=\operatorname {var} (X)=\operatorname {E} {\big (}(X-\operatorname {E} (X))^{2}{\big )}={}}$the variance, or second central moment.
• ${\textstyle \kappa _{3}(X)=\operatorname {E} {\big (}(X-\operatorname {E} (X))^{3}{\big )}={}}$the third central moment.
• ${\textstyle \kappa _{4}(X)=\operatorname {E} {\big (}(X-\operatorname {E} (X))^{4}{\big )}-3\left(\operatorname {E} {\big (}(X-\operatorname {E} (X))^{2}{\big )}\right)^{2}={}}$the fourth central moment minus three times the square of the second central moment. Thus this is the first case in which cumulants are not simply moments or central moments. The central moments of degree more than 3 lack the cumulative property.
• ${\textstyle \kappa _{5}(X)=\operatorname {E} {\big (}(X-\operatorname {E} (X))^{5}{\big )}-10\operatorname {E} {\big (}(X-\operatorname {E} (X))^{3}{\big )}\operatorname {E} {\big (}(X-\operatorname {E} (X))^{2}{\big )}.}$

Given the results for the cumulants of the normal distribution, it might be hoped to find families of distributions for which κ_m = κ_m+1 = ⋯ = 0 for some m > 3, with the lower-order cumulants (orders 3 to m − 1) being non-zero. There are no such distributions.[7] The underlying result here is that the cumulant generating function cannot be a finite-order polynomial of degree greater than 2.

The moment generating function is given by:

${\displaystyle M(t)=1+\sum _{n=1}^{\infty }{\frac {\mu '_{n}t^{n}}{n!}}=\exp \left(\sum _{n=1}^{\infty }{\frac {\kappa _{n}t^{n}}{n!}}\right)=\exp(K(t)).}$

So the cumulant generating function is the logarithm of the moment generating function

${\displaystyle K(t)=\log M(t).}$

The first cumulant is the expected value; the second and third cumulants are respectively the second and third central moments (the second central moment is the variance); but the higher cumulants are neither moments nor central moments, but rather more complicated polynomial functions of the moments.

The moments can be recovered in terms of cumulants by evaluating the n-th derivative of ${\displaystyle \exp(K(t))}$ at ${\displaystyle t=0}$,

${\displaystyle \mu '_{n}=M^{(n)}(0)=\left.{\frac {\mathrm {d} ^{n}\exp(K(t))}{\mathrm {d} t^{n}}}\right|_{t=0}.}$

Likewise, the cumulants can be recovered in terms of moments by evaluating the n-th derivative of ${\displaystyle \log M(t)}$ at ${\displaystyle t=0}$,

${\displaystyle \kappa _{n}=K^{(n)}(0)=\left.{\frac {\mathrm {d} ^{n}\log M(t)}{\mathrm {d} t^{n}}}\right|_{t=0}.}$

The explicit expression for the n-th moment in terms of the first n cumulants, and vice versa, can be obtained by using Faà di Bruno's formula for higher derivatives of composite functions. In general, we have

${\displaystyle \mu '_{n}=\sum _{k=1}^{n}B_{n,k}(\kappa _{1},\ldots ,\kappa _{n-k+1})}$

${\displaystyle \kappa _{n}=\sum _{k=1}^{n}(-1)^{k-1}(k-1)!B_{n,k}(\mu '_{1},\ldots ,\mu '_{n-k+1}),}$

where ${\displaystyle B_{n,k}}$ are incomplete (or partial) Bell polynomials.

In the like manner, if the mean is given by ${\displaystyle \mu }$, the central moment generating function is given by

${\displaystyle C(t)=\operatorname {E} [e^{t(x-\mu )}]=e^{-\mu t}M(t)=\exp(K(t)-\mu t),}$

and the n-th central moment is obtained in terms of cumulants as

${\displaystyle \mu _{n}=C^{(n)}(0)=\left.{\frac {\mathrm {d} ^{n}}{\mathrm {d} t^{n}}}\exp(K(t)-\mu t)\right|_{t=0}=\sum _{k=1}^{n}B_{n,k}(0,\kappa _{2},\ldots ,\kappa _{n-k+1}).}$

Also, for n > 1, the n-th cumulant in terms of the central moments is

${\displaystyle {\begin{aligned}\kappa _{n}&=K^{(n)}(0)=\left.{\frac {\mathrm {d} ^{n}}{\mathrm {d} t^{n}}}(\log C(t)+\mu t)\right|_{t=0}\\[4pt]&=\sum _{k=1}^{n}(-1)^{k-1}(k-1)!B_{n,k}(0,\mu _{2},\ldots ,\mu _{n-k+1}).\end{aligned}}}$

The n-th moment μ′_n is an n-th-degree polynomial in the first n cumulants. The first few expressions are:

${\displaystyle {\begin{aligned}\mu '_{1}={}&\kappa _{1}\\[5pt]\mu '_{2}={}&\kappa _{2}+\kappa _{1}^{2}\\[5pt]\mu '_{3}={}&\kappa _{3}+3\kappa _{2}\kappa _{1}+\kappa _{1}^{3}\\[5pt]\mu '_{4}={}&\kappa _{4}+4\kappa _{3}\kappa _{1}+3\kappa _{2}^{2}+6\kappa _{2}\kappa _{1}^{2}+\kappa _{1}^{4}\\[5pt]\mu '_{5}={}&\kappa _{5}+5\kappa _{4}\kappa _{1}+10\kappa _{3}\kappa _{2}+10\kappa _{3}\kappa _{1}^{2}+15\kappa _{2}^{2}\kappa _{1}+10\kappa _{2}\kappa _{1}^{3}+\kappa _{1}^{5}\\[5pt]\mu '_{6}={}&\kappa _{6}+6\kappa _{5}\kappa _{1}+15\kappa _{4}\kappa _{2}+15\kappa _{4}\kappa _{1}^{2}+10\kappa _{3}^{2}+60\kappa _{3}\kappa _{2}\kappa _{1}+20\kappa _{3}\kappa _{1}^{3}\\&{}+15\kappa _{2}^{3}+45\kappa _{2}^{2}\kappa _{1}^{2}+15\kappa _{2}\kappa _{1}^{4}+\kappa _{1}^{6}.\end{aligned}}}$

The "prime" distinguishes the moments μ′_n from the central moments μ_n. To express the central moments as functions of the cumulants, just drop from these polynomials all terms in which κ₁ appears as a factor:

${\displaystyle {\begin{aligned}\mu _{1}&=0\\[4pt]\mu _{2}&=\kappa _{2}\\[4pt]\mu _{3}&=\kappa _{3}\\[4pt]\mu _{4}&=\kappa _{4}+3\kappa _{2}^{2}\\[4pt]\mu _{5}&=\kappa _{5}+10\kappa _{3}\kappa _{2}\\[4pt]\mu _{6}&=\kappa _{6}+15\kappa _{4}\kappa _{2}+10\kappa _{3}^{2}+15\kappa _{2}^{3}.\end{aligned}}}$

Similarly, the n-th cumulant κ_n is an n-th-degree polynomial in the first n non-central moments. The first few expressions are:

${\displaystyle {\begin{aligned}\kappa _{1}={}&\mu '_{1}\\[4pt]\kappa _{2}={}&\mu '_{2}-{\mu '_{1}}^{2}\\[4pt]\kappa _{3}={}&\mu '_{3}-3\mu '_{2}\mu '_{1}+2{\mu '_{1}}^{3}\\[4pt]\kappa _{4}={}&\mu '_{4}-4\mu '_{3}\mu '_{1}-3{\mu '_{2}}^{2}+12\mu '_{2}{\mu '_{1}}^{2}-6{\mu '_{1}}^{4}\\[4pt]\kappa _{5}={}&\mu '_{5}-5\mu '_{4}\mu '_{1}-10\mu '_{3}\mu '_{2}+20\mu '_{3}{\mu '_{1}}^{2}+30{\mu '_{2}}^{2}\mu '_{1}-60\mu '_{2}{\mu '_{1}}^{3}+24{\mu '_{1}}^{5}\\[4pt]\kappa _{6}={}&\mu '_{6}-6\mu '_{5}\mu '_{1}-15\mu '_{4}\mu '_{2}+30\mu '_{4}{\mu '_{1}}^{2}-10{\mu '_{3}}^{2}+120\mu '_{3}\mu '_{2}\mu '_{1}\\&{}-120\mu '_{3}{\mu '_{1}}^{3}+30{\mu '_{2}}^{3}-270{\mu '_{2}}^{2}{\mu '_{1}}^{2}+360\mu '_{2}{\mu '_{1}}^{4}-120{\mu '_{1}}^{6}\,.\end{aligned}}}$

In general,[8] the cumulant is the determinant of a matrix:

$${\displaystyle \kappa _{l}=(-1)^{l+1}\left|{\begin{array}{cccccccc}\mu '_{1}&1&0&0&0&0&\ldots &0\\\mu '_{2}&\mu '_{1}&1&0&0&0&\ldots &0\\\mu '_{3}&\mu '_{2}&\left({\begin{array}{l}2\\1\end{array}}\right)\mu '_{1}&1&0&0&\ldots &0\\\mu '_{4}&\mu '_{3}&\left({\begin{array}{l}3\\1\end{array}}\right)\mu '_{2}&\left({\begin{array}{l}3\\2\end{array}}\right)\mu '_{1}&1&0&\ldots &0\\\mu '_{5}&\mu '_{4}&\left({\begin{array}{l}4\\1\end{array}}\right)\mu '_{3}&\left({\begin{array}{l}4\\2\end{array}}\right)\mu '_{2}&\left({\begin{array}{c}4\\3\end{array}}\right)\mu '_{1}&1&\ldots &0\\\vdots &\vdots &\vdots &\vdots &\vdots &\ddots &\ddots &\vdots \\\mu '_{l-1}&\mu '_{l-2}&\ldots &\ldots &\ldots &\ldots &\ddots &1\\\mu '_{l}&\mu '_{l-1}&\ldots &\ldots &\ldots &\ldots &\ldots &\left({\begin{array}{l}l-1\\l-2\end{array}}\right)\mu '_{1}\end{array}}\right|}$$

To express the cumulants κ_n for n > 1 as functions of the central moments, drop from these polynomials all terms in which μ'₁ appears as a factor:

${\displaystyle \kappa _{2}=\mu _{2}\,}$

${\displaystyle \kappa _{3}=\mu _{3}\,}$

${\displaystyle \kappa _{4}=\mu _{4}-3{\mu _{2}}^{2}\,}$

${\displaystyle \kappa _{5}=\mu _{5}-10\mu _{3}\mu _{2}\,}$

${\displaystyle \kappa _{6}=\mu _{6}-15\mu _{4}\mu _{2}-10{\mu _{3}}^{2}+30{\mu _{2}}^{3}\,.}$

To express the cumulants κ_n for n > 2 as functions of the standardized central moments μ″_n, also set μ'₂=1 in the polynomials:

${\displaystyle \kappa _{3}=\mu ''_{3}\,}$

${\displaystyle \kappa _{4}=\mu ''_{4}-3\,}$

${\displaystyle \kappa _{5}=\mu ''_{5}-10\mu ''_{3}\,}$

${\displaystyle \kappa _{6}=\mu ''_{6}-15\mu ''_{4}-10{\mu ''_{3}}^{2}+30\,.}$

The cumulants can be related to the moments by differentiating the relationship log M(t) = K(t) with respect to t, giving M′(t) = K′(t) M(t), which conveniently contains no exponentials or logarithms. Equating the coefficient of t ⁿ⁻¹ / (n−1)! on the left and right sides and using μ′₀ = 1 gives the following formulas for n ≥ 1:[9]

${\displaystyle {\begin{aligned}\mu '_{1}={}&\kappa _{1}\\[1pt]\mu '_{2}={}&\kappa _{1}\mu '_{1}+\kappa _{2}\\[1pt]\mu '_{3}={}&\kappa _{1}\mu '_{2}+2\kappa _{2}\mu '_{1}+\kappa _{3}\\[1pt]\mu '_{4}={}&\kappa _{1}\mu '_{3}+3\kappa _{2}\mu '_{2}+3\kappa _{3}\mu '_{1}+\kappa _{4}\\[1pt]\mu '_{5}={}&\kappa _{1}\mu '_{4}+4\kappa _{2}\mu '_{3}+6\kappa _{3}\mu '_{2}+4\kappa _{4}\mu '_{1}+\kappa _{5}\\[1pt]\mu '_{6}={}&\kappa _{1}\mu '_{5}+5\kappa _{2}\mu '_{4}+10\kappa _{3}\mu '_{3}+10\kappa _{4}\mu '_{2}+5\kappa _{5}\mu '_{1}+\kappa _{6}\\[1pt]\mu '_{n}={}&\sum _{m=1}^{n-1}{n-1 \choose m-1}\kappa _{m}\mu '_{n-m}+\kappa _{n}\,.\end{aligned}}}$

These allow either ${\displaystyle \kappa _{n}}$ or ${\displaystyle \mu '_{n}}$ to be computed from the other using knowledge of the lower-order cumulants and moments. The corresponding formulas for the central moments ${\displaystyle \mu _{n}}$ for ${\displaystyle n\geq 2}$ are formed from these formulas by setting ${\displaystyle \mu '_{1}=\kappa _{1}=0}$ and replacing each ${\displaystyle \mu '_{n}}$ with ${\displaystyle \mu _{n}}$ for ${\displaystyle n\geq 2}$:

${\displaystyle {\begin{aligned}\mu _{2}={}&\kappa _{2}\\[1pt]\mu _{3}={}&\kappa _{3}\\[1pt]\mu _{n}={}&\sum _{m=2}^{n-2}{n-1 \choose m-1}\kappa _{m}\mu _{n-m}+\kappa _{n}\,.\end{aligned}}}$

These polynomials have a remarkable combinatorial interpretation: the coefficients count certain partitions of sets. A general form of these polynomials is

${\displaystyle \mu '_{n}=\sum _{\pi \,\in \,\Pi }\prod _{B\,\in \,\pi }\kappa _{|B|}}$

where

• π runs through the list of all partitions of a set of size n;
• "B ∈ π" means B is one of the "blocks" into which the set is partitioned; and
• |B| is the size of the set B.

Thus each monomial is a constant times a product of cumulants in which the sum of the indices is n (e.g., in the term κ₃ κ₂² κ₁, the sum of the indices is 3 + 2 + 2 + 1 = 8; this appears in the polynomial that expresses the 8th moment as a function of the first eight cumulants). A partition of the integer n corresponds to each term. The coefficient in each term is the number of partitions of a set of n members that collapse to that partition of the integer n when the members of the set become indistinguishable.

Further connection between cumulants and combinatorics can be found in the work of Gian-Carlo Rota, where links to invariant theory, symmetric functions, and binomial sequences are studied via umbral calculus.[10]

The law of total expectation and the law of total variance generalize naturally to conditional cumulants. The case n = 3, expressed in the language of (central) moments rather than that of cumulants, says

${\displaystyle \mu _{3}(X)=\operatorname {E} (\mu _{3}(X\mid Y))+\mu _{3}(\operatorname {E} (X\mid Y))+3\operatorname {cov} (\operatorname {E} (X\mid Y),\operatorname {var} (X\mid Y)).}$

In general,[11]

${\displaystyle \kappa (X_{1},\dots ,X_{n})=\sum _{\pi }\kappa (\kappa (X_{\pi _{1}}\mid Y),\dots ,\kappa (X_{\pi _{b}}\mid Y))}$

where

• the sum is over all partitions π of the set { 1, ..., n } of indices, and
• π₁, ..., π_b are all of the "blocks" of the partition π; the expression κ(X_πm) indicates that the joint cumulant of the random variables whose indices are in that block of the partition.

For certain settings, a derivative identity can be established between the conditional cumulant and the conditional cumulant. For example, suppose that Y = X + Z where Z is standard normal independent of X, then for any X it holds that[12]

${\displaystyle \kappa _{n+1}(X\mid Y=y)={\frac {\mathrm {d} ^{n}}{\mathrm {d} y^{n}}}\operatorname {E} (X\mid Y=y),\,n\in \mathbb {N} ,\,y\in \mathbb {R} .}$

The results can also be texted to the exponential family.[13]

More generally, the cumulants of a sequence { m_n : n = 1, 2, 3, ... }, not necessarily the moments of any probability distribution, are, by definition,

${\displaystyle 1+\sum _{n=1}^{\infty }{\frac {m_{n}t^{n}}{n!}}=\exp \left(\sum _{n=1}^{\infty }{\frac {\kappa _{n}t^{n}}{n!}}\right),}$

where the values of κ_n for n = 1, 2, 3, ... are found formally, i.e., by algebra alone, in disregard of questions of whether any series converges. All of the difficulties of the "problem of cumulants" are absent when one works formally. The simplest example is that the second cumulant of a probability distribution must always be nonnegative, and is zero only if all of the higher cumulants are zero. Formal cumulants are subject to no such constraints.

In combinatorics, the n-th Bell number is the number of partitions of a set of size n. All of the cumulants of the sequence of Bell numbers are equal to 1. The Bell numbers are the moments of the Poisson distribution with expected value 1.

For any sequence { κ_n : n = 1, 2, 3, ... } of scalars in a field of characteristic zero, being considered formal cumulants, there is a corresponding sequence { μ ′ : n = 1, 2, 3, ...} of formal moments, given by the polynomials above. For those polynomials, construct a polynomial sequence in the following way. Out of the polynomial

${\displaystyle {\begin{aligned}\mu '_{6}={}&\kappa _{6}+6\kappa _{5}\kappa _{1}+15\kappa _{4}\kappa _{2}+15\kappa _{4}\kappa _{1}^{2}+10\kappa _{3}^{2}+60\kappa _{3}\kappa _{2}\kappa _{1}+20\kappa _{3}\kappa _{1}^{3}\\&{}+15\kappa _{2}^{3}+45\kappa _{2}^{2}\kappa _{1}^{2}+15\kappa _{2}\kappa _{1}^{4}+\kappa _{1}^{6}\end{aligned}}}$

make a new polynomial in these plus one additional variable x:

${\displaystyle {\begin{aligned}p_{6}(x)={}&\kappa _{6}\,x+(6\kappa _{5}\kappa _{1}+15\kappa _{4}\kappa _{2}+10\kappa _{3}^{2})\,x^{2}+(15\kappa _{4}\kappa _{1}^{2}+60\kappa _{3}\kappa _{2}\kappa _{1}+15\kappa _{2}^{3})\,x^{3}\\&{}+(45\kappa _{2}^{2}\kappa _{1}^{2})\,x^{4}+(15\kappa _{2}\kappa _{1}^{4})\,x^{5}+(\kappa _{1}^{6})\,x^{6},\end{aligned}}}$

and then generalize the pattern. The pattern is that the numbers of blocks in the aforementioned partitions are the exponents on x. Each coefficient is a polynomial in the cumulants; these are the Bell polynomials, named after Eric Temple Bell.

This sequence of polynomials is of binomial type. In fact, no other sequences of binomial type exist; every polynomial sequence of binomial type is completely determined by its sequence of formal cumulants.

In the above moment-cumulant formula

${\displaystyle \operatorname {E} (X_{1}\cdots X_{n})=\sum _{\pi }\prod _{B\,\in \,\pi }\kappa (X_{i}:i\in B)}$

for joint cumulants, one sums over all partitions of the set { 1, ..., n }. If instead, one sums only over the noncrossing partitions, then, by solving these formulae for the ${\displaystyle \kappa }$ in terms of the moments, one gets free cumulants rather than conventional cumulants treated above. These free cumulants were introduced by Roland Speicher and play a central role in free probability theory.[20][21] In that theory, rather than considering independence of random variables, defined in terms of tensor products of algebras of random variables, one considers instead free independence of random variables, defined in terms of free products of algebras.[21]

The ordinary cumulants of degree higher than 2 of the normal distribution are zero. The free cumulants of degree higher than 2 of the Wigner semicircle distribution are zero.[21] This is one respect in which the role of the Wigner distribution in free probability theory is analogous to that of the normal distribution in conventional probability theory.