Kurtosis

The exact interpretation of the Pearson measure of kurtosis (or excess kurtosis) used to be disputed, but is now settled. As Westfall notes in 2014[2], "...its only unambiguous interpretation is in terms of tail extremity; i.e., either existing outliers (for the sample kurtosis) or propensity to produce outliers (for the kurtosis of a probability distribution)." The logic is simple: Kurtosis is the average (or expected value) of the standardized data raised to the fourth power. Standardized values that are less than 1 (i.e., data within one standard deviation of the mean, where the "peak" would be) contribute virtually nothing to kurtosis, since raising a number that is less than 1 to the fourth power makes it closer to zero. The only data values (observed or observable) that contribute to kurtosis in any meaningful way are those outside the region of the peak; i.e., the outliers. Therefore, kurtosis measures outliers only; it measures nothing about the "peak".

Many incorrect interpretations of kurtosis that involve notions of peakedness have been given. One is that kurtosis measures both the "peakedness" of the distribution and the heaviness of its tail.[5] Various other incorrect interpretations have been suggested, such as "lack of shoulders" (where the "shoulder" is defined vaguely as the area between the peak and the tail, or more specifically as the area about one standard deviation from the mean) or "bimodality".[6] Balanda and MacGillivray assert that the standard definition of kurtosis "is a poor measure of the kurtosis, peakedness, or tail weight of a distribution"[5]^: 114 and instead propose to "define kurtosis vaguely as the location- and scale-free movement of probability mass from the shoulders of a distribution into its center and tails".[5]

In 1986 Moors gave an interpretation of kurtosis.[7] Let

${\displaystyle Z={\frac {X-\mu }{\sigma }},}$

where X is a random variable, μ is the mean and σ is the standard deviation.

Now by definition of the kurtosis ${\displaystyle \kappa }$, and by the well-known identity ${\displaystyle E\left[V^{2}\right]=\operatorname {var} [V]+[E[V]]^{2},}$

${\displaystyle \kappa =E\left[Z^{4}\right]=\operatorname {var} \left[Z^{2}\right]+\left[E\left[Z^{2}\right]\right]^{2}=\operatorname {var} \left[Z^{2}\right]+[\operatorname {var} [Z]]^{2}=\operatorname {var} \left[Z^{2}\right]+1}$.

The kurtosis can now be seen as a measure of the dispersion of Z² around its expectation. Alternatively it can be seen to be a measure of the dispersion of Z around +1 and −1. κ attains its minimal value in a symmetric two-point distribution. In terms of the original variable X, the kurtosis is a measure of the dispersion of X around the two values μ ± σ.

High values of κ arise in two circumstances:

• where the probability mass is concentrated around the mean and the data-generating process produces occasional values far from the mean,
• where the probability mass is concentrated in the tails of the distribution.

The entropy of a distribution is ${\displaystyle \int p(x)\ln p(x)dx}$.

For any ${\displaystyle \mu \in \mathbb {R} ^{n},\Sigma \in \mathbb {R} ^{n\times n}}$ with ${\displaystyle \Sigma }$ positive definite, among all probability distributions on ${\displaystyle \mathbb {R} ^{n}}$ with mean ${\displaystyle \mu }$ and covariance ${\displaystyle \Sigma }$, the normal distribution ${\displaystyle {\mathcal {N}}(\mu ,\Sigma )}$ has the largest entropy.

Since mean ${\displaystyle \mu }$ and covariance ${\displaystyle \Sigma }$ are the first two moments, it is natural to consider extension to higher moments. In fact, by Lagrange multiplier method, for any prescribed first n moments, if there exists some probability distribution of form ${\displaystyle p(x)\propto e^{\sum _{i}a_{i}x_{i}+\sum _{ij}b_{ij}x_{i}x_{j}+\cdots +\sum _{i_{1}\cdots i_{n}}x_{i_{1}}\cdots x_{i_{n}}}}$ that has the prescribed moments (if it is feasible), then it is the maximal entropy distribution under the given constraints.[8][9]

By serial expansion,

$${\displaystyle \int {\frac {1}{\sqrt {2\pi }}}e^{-{\frac {1}{2}}x^{2}-{\frac {1}{4}}gx^{4}}x^{2n}dx={\frac {1}{\sqrt {2\pi }}}\int e^{-{\frac {1}{2}}x^{2}-{\frac {1}{4}}gx^{4}}x^{2n}dx=\sum _{k}{\frac {1}{k!}}(-g/4)^{k}(2n+4k-1)!!=(2n-1)!!-{\frac {1}{4}}g(2n+3)!!+O(g^{2})}$$

so if a random variable has probability distribution ${\displaystyle p(x)=e^{-{\frac {1}{2}}x^{2}-{\frac {1}{4}}gx^{4}}/Z}$, where ${\displaystyle Z}$ is a normalization constant, then its kurtosis is ${\displaystyle 3-6g+O(g^{2})}$.[10]

Distributions with zero excess kurtosis are called mesokurtic, or mesokurtotic. The most prominent example of a mesokurtic distribution is the normal distribution family, regardless of the values of its parameters. A few other well-known distributions can be mesokurtic, depending on parameter values: for example, the binomial distribution is mesokurtic for ${\displaystyle p=1/2\pm {\sqrt {1/12}}}$.

A distribution with positive excess kurtosis is called leptokurtic, or leptokurtotic. "Lepto-" means "slender".[11] In terms of shape, a leptokurtic distribution has fatter tails. Examples of leptokurtic distributions include the Student's t-distribution, Rayleigh distribution, Laplace distribution, exponential distribution, Poisson distribution and the logistic distribution. Such distributions are sometimes termed super-Gaussian.[12]

The coin toss is the most platykurtic distribution

A distribution with negative excess kurtosis is called platykurtic, or platykurtotic. "Platy-" means "broad".[13] In terms of shape, a platykurtic distribution has thinner tails. Examples of platykurtic distributions include the continuous and discrete uniform distributions, and the raised cosine distribution. The most platykurtic distribution of all is the Bernoulli distribution with p = 1/2 (for example the number of times one obtains "heads" when flipping a coin once, a coin toss), for which the excess kurtosis is −2.

pdf for the Pearson type VII distribution with excess kurtosis of infinity (red); 2 (blue); and 0 (black)

log-pdf for the Pearson type VII distribution with excess kurtosis of infinity (red); 2 (blue); 1, 1/2, 1/4, 1/8, and 1/16 (gray); and 0 (black)

The effects of kurtosis are illustrated using a parametric family of distributions whose kurtosis can be adjusted while their lower-order moments and cumulants remain constant. Consider the Pearson type VII family, which is a special case of the Pearson type IV family restricted to symmetric densities. The probability density function is given by

${\displaystyle f(x;a,m)={\frac {\Gamma (m)}{a\,{\sqrt {\pi }}\,\Gamma (m-1/2)}}\left[1+\left({\frac {x}{a}}\right)^{2}\right]^{-m},\!}$

where a is a scale parameter and m is a shape parameter.

All densities in this family are symmetric. The kth moment exists provided m > (k + 1)/2. For the kurtosis to exist, we require m > 5/2. Then the mean and skewness exist and are both identically zero. Setting a² = 2m − 3 makes the variance equal to unity. Then the only free parameter is m, which controls the fourth moment (and cumulant) and hence the kurtosis. One can reparameterize with ${\displaystyle m=5/2+3/\gamma _{2}}$, where ${\displaystyle \gamma _{2}}$ is the excess kurtosis as defined above. This yields a one-parameter leptokurtic family with zero mean, unit variance, zero skewness, and arbitrary non-negative excess kurtosis. The reparameterized density is

${\displaystyle g(x;\gamma _{2})=f\left(x;\;a={\sqrt {2+{\frac {6}{\gamma _{2}}}}},\;m={\frac {5}{2}}+{\frac {3}{\gamma _{2}}}\right).\!}$

In the limit as ${\displaystyle \gamma _{2}\to \infty }$ one obtains the density

${\displaystyle g(x)=3\left(2+x^{2}\right)^{-{\frac {5}{2}}},\!}$

which is shown as the red curve in the images on the right.

In the other direction as ${\displaystyle \gamma _{2}\to 0}$ one obtains the standard normal density as the limiting distribution, shown as the black curve.

In the images on the right, the blue curve represents the density ${\displaystyle x\mapsto g(x;2)}$ with excess kurtosis of 2. The top image shows that leptokurtic densities in this family have a higher peak than the mesokurtic normal density, although this conclusion is only valid for this select family of distributions. The comparatively fatter tails of the leptokurtic densities are illustrated in the second image, which plots the natural logarithm of the Pearson type VII densities: the black curve is the logarithm of the standard normal density, which is a parabola. One can see that the normal density allocates little probability mass to the regions far from the mean ("has thin tails"), compared with the blue curve of the leptokurtic Pearson type VII density with excess kurtosis of 2. Between the blue curve and the black are other Pearson type VII densities with γ₂ = 1, 1/2, 1/4, 1/8, and 1/16. The red curve again shows the upper limit of the Pearson type VII family, with ${\displaystyle \gamma _{2}=\infty }$ (which, strictly speaking, means that the fourth moment does not exist). The red curve decreases the slowest as one moves outward from the origin ("has fat tails").

Probability density functions for selected distributions with mean 0, variance 1 and different excess kurtosis

Logarithms of probability density functions for selected distributions with mean 0, variance 1 and different excess kurtosis

Several well-known, unimodal, and symmetric distributions from different parametric families are compared here. Each has a mean and skewness of zero. The parameters have been chosen to result in a variance equal to 1 in each case. The images on the right show curves for the following seven densities, on a linear scale and logarithmic scale:

• D: Laplace distribution, also known as the double exponential distribution, red curve (two straight lines in the log-scale plot), excess kurtosis = 3
• S: hyperbolic secant distribution, orange curve, excess kurtosis = 2
• L: logistic distribution, green curve, excess kurtosis = 1.2
• N: normal distribution, black curve (inverted parabola in the log-scale plot), excess kurtosis = 0
• C: raised cosine distribution, cyan curve, excess kurtosis = −0.593762...
• W: Wigner semicircle distribution, blue curve, excess kurtosis = −1
• U: uniform distribution, magenta curve (shown for clarity as a rectangle in both images), excess kurtosis = −1.2.

Note that in these cases the platykurtic densities have bounded support, whereas the densities with positive or zero excess kurtosis are supported on the whole real line.

One cannot infer that high or low kurtosis distributions have the characteristics indicated by these examples. There exist platykurtic densities with infinite support,

• e.g., exponential power distributions with sufficiently large shape parameter b

and there exist leptokurtic densities with finite support.

• e.g., a distribution that is uniform between −3 and −0.3, between −0.3 and 0.3, and between 0.3 and 3, with the same density in the (−3, −0.3) and (0.3, 3) intervals, but with 20 times more density in the (−0.3, 0.3) interval

Also, there exist platykurtic densities with infinite peakedness,

• e.g., an equal mixture of the beta distribution with parameters 0.5 and 1 with its reflection about 0.0

and there exist leptokurtic densities that appear flat-topped,

• e.g., a mixture of distribution that is uniform between -1 and 1 with a T(4.0000001) Student's t-distribution, with mixing probabilities 0.999 and 0.001.

For a sample of n values, a method of moments estimator of the population excess kurtosis can be defined as

${\displaystyle g_{2}={\frac {m_{4}}{m_{2}^{2}}}-3={\frac {{\tfrac {1}{n}}\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{4}}{\left[{\tfrac {1}{n}}\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{2}\right]^{2}}}-3}$

where m₄ is the fourth sample moment about the mean, m₂ is the second sample moment about the mean (that is, the sample variance), x_i is the i^th value, and ${\displaystyle {\overline {x}}}$ is the sample mean.

This formula has the simpler representation,

${\displaystyle g_{2}={\frac {1}{n}}\sum _{i=1}^{n}z_{i}^{4}-3}$

where the ${\displaystyle z_{i}}$ values are the standardized data values using the standard deviation defined using n rather than n − 1 in the denominator.

For example, suppose the data values are 0, 3, 4, 1, 2, 3, 0, 2, 1, 3, 2, 0, 2, 2, 3, 2, 5, 2, 3, 999.

Then the ${\displaystyle z_{i}}$ values are −0.239, −0.225, −0.221, −0.234, −0.230, −0.225, −0.239, −0.230, −0.234, −0.225, −0.230, −0.239, −0.230, −0.230, −0.225, −0.230, −0.216, −0.230, −0.225, 4.359

and the ${\displaystyle z_{i}^{4}}$ values are 0.003, 0.003, 0.002, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.002, 0.003, 0.003, 360.976.

The average of these values is 18.05 and the excess kurtosis is thus 18.05 − 3 = 15.05. This example makes it clear that data near the "middle" or "peak" of the distribution do not contribute to the kurtosis statistic, hence kurtosis does not measure "peakedness". It is simply a measure of the outlier, 999 in this example.

Given a sub-set of samples from a population, the sample excess kurtosis ${\displaystyle g_{2}}$ above is a biased estimator of the population excess kurtosis. An alternative estimator of the population excess kurtosis, which is unbiased in random samples of a normal distribution, is defined as follows:[3]

${\displaystyle {\begin{aligned}G_{2}&={\frac {k_{4}}{k_{2}^{2}}}\\[6pt]&={\frac {n^{2}\,[(n+1)\,m_{4}-3\,(n-1)\,m_{2}^{2}]}{(n-1)\,(n-2)\,(n-3)}}\;{\frac {(n-1)^{2}}{n^{2}\,m_{2}^{2}}}\\[6pt]&={\frac {n-1}{(n-2)\,(n-3)}}\left[(n+1)\,{\frac {m_{4}}{m_{2}^{2}}}-3\,(n-1)\right]\\[6pt]&={\frac {n-1}{(n-2)\,(n-3)}}\left[(n+1)\,g_{2}+6\right]\\[6pt]&={\frac {(n+1)\,n\,(n-1)}{(n-2)\,(n-3)}}\;{\frac {\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{4}}{\left(\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}\right)^{2}}}-3\,{\frac {(n-1)^{2}}{(n-2)\,(n-3)}}\\[6pt]&={\frac {(n+1)\,n}{(n-1)\,(n-2)\,(n-3)}}\;{\frac {\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{4}}{k_{2}^{2}}}-3\,{\frac {(n-1)^{2}}{(n-2)(n-3)}}\end{aligned}}}$

where k₄ is the unique symmetric unbiased estimator of the fourth cumulant, k₂ is the unbiased estimate of the second cumulant (identical to the unbiased estimate of the sample variance), m₄ is the fourth sample moment about the mean, m₂ is the second sample moment about the mean, x_i is the i^th value, and ${\displaystyle {\bar {x}}}$ is the sample mean. This adjusted Fisher–Pearson standardized moment coefficient ${\displaystyle G_{2}}$ is the version found in Excel and several statistical packages including Minitab, SAS, and SPSS.[14]

Unfortunately, in nonnormal samples ${\displaystyle G_{2}}$ is itself generally biased.

An upper bound for the sample kurtosis of n (n > 2) real numbers is[15]

${\displaystyle g_{2}\leq {\frac {1}{2}}{\frac {n-3}{n-2}}g_{1}^{2}+{\frac {n}{2}}-3.}$

where ${\displaystyle g_{1}=m_{3}/m_{2}^{3/2}}$ is the corresponding sample skewness.

The variance of the sample kurtosis of a sample of size n from the normal distribution is[16]

${\displaystyle \operatorname {var} (g_{2})={\frac {24n(n-1)^{2}}{(n-3)(n-2)(n+3)(n+5)}}}$

Stated differently, under the assumption that the underlying random variable ${\displaystyle X}$ is normally distributed, it can be shown that ${\displaystyle {\sqrt {n}}g_{2}{\xrightarrow {d}}{\mathcal {N}}(0,24)}$.[17]^{: Page number needed}

Applying band-pass filters to digital images, kurtosis values tend to be uniform, independent of the range of the filter. This behavior, termed kurtosis convergence, can be used to detect image splicing in forensic analysis.[21]