Pearson's chi-squared test

Two cells

In the special case where there are only two cells in the table, the expected values follow a binomial distribution,

${\displaystyle O\ \sim \ {\mbox{Bin}}(n,p),\,}$

where

p = probability, under the null hypothesis,

n = number of observations in the sample.

In the above example the hypothesised probability of a male observation is 0.5, with 100 samples. Thus we expect to observe 50 males.

If n is sufficiently large, the above binomial distribution may be approximated by a Gaussian (normal) distribution and thus the Pearson test statistic approximates a chi-squared distribution,

${\displaystyle {\text{Bin}}(n,p)\approx {\text{N}}(np,np(1-p)).\,}$

Let O₁ be the number of observations from the sample that are in the first cell. The Pearson test statistic can be expressed as

${\displaystyle {\frac {(O_{1}-np)^{2}}{np}}+{\frac {(n-O_{1}-n(1-p))^{2}}{n(1-p)}},}$

which can in turn be expressed as

${\displaystyle \left({\frac {O_{1}-np}{\sqrt {np(1-p)}}}\right)^{2}.}$

By the normal approximation to a binomial this is the squared of one standard normal variate, and hence is distributed as chi-squared with 1 degree of freedom. Note that the denominator is one standard deviation of the Gaussian approximation, so can be written

${\displaystyle {\frac {(O_{1}-\mu )^{2}}{\sigma ^{2}}}.}$

So as consistent with the meaning of the chi-squared distribution, we are measuring how probable the observed number of standard deviations away from the mean is under the Gaussian approximation (which is a good approximation for large n).

The chi-squared distribution is then integrated on the right of the statistic value to obtain the P-value, which is equal to the probability of getting a statistic equal or bigger than the observed one, assuming the null hypothesis.

Two-by-two contingency tables

When the test is applied to a contingency table containing two rows and two columns, the test is equivalent to a Z-test of proportions.

Many cells

Broadly similar arguments as above lead to the desired result, though the details are more involved. One may apply an orthogonal change of variables to turn the limiting summands in the test statistic into one fewer squares of i.i.d. standard normal random variables.[11]

Let us now prove that the distribution indeed approaches asymptotically the ${\displaystyle \chi ^{2}}$ distribution as the number of observations approaches infinity.

Let ${\displaystyle n}$ be the number of observations, ${\displaystyle m}$ the number of cells and ${\displaystyle p_{i}}$ the probability of an observation to fall in the i-th cell, for ${\displaystyle 1\leq i\leq m}$. We denote by ${\displaystyle \{k_{i}\}}$ the configuration where for each i there are ${\displaystyle k_{i}}$ observations in the i-th cell. Note that

${\displaystyle \sum _{i=1}^{m}k_{i}=n\qquad {\text{and}}\qquad \sum _{i=1}^{m}p_{i}=1.}$

Let ${\displaystyle \chi _{P}^{2}(\{k_{i}\},\{p_{i}\})}$ be Pearson's cumulative test statistic for such a configuration, and let ${\displaystyle \chi _{P}^{2}(\{p_{i}\})}$ be the distribution of this statistic. We will show that the latter probability approaches the ${\displaystyle \chi ^{2}}$ distribution with ${\displaystyle m-1}$ degrees of freedom, as ${\displaystyle n\to \infty .}$

For any arbitrary value T:

${\displaystyle P(\chi _{P}^{2}(\{p_{i}\})>T)=\sum _{\{k_{i}\}|\chi _{P}^{2}(\{k_{i}\},\{p_{i}\})>T}{\frac {n!}{k_{1}!\cdots k_{m}!}}\prod _{i=1}^{m}{p_{i}}^{k_{i}}}$

We will use a procedure similar to the approximation in de Moivre–Laplace theorem. Contributions from small ${\displaystyle k_{i}}$ are of subleading order in ${\displaystyle n}$ and thus for large ${\displaystyle n}$ we may use Stirling's formula for both ${\displaystyle n!}$ and ${\displaystyle k_{i}!}$ to get the following:

${\displaystyle P(\chi _{P}^{2}(\{p_{i}\})>T)\sim \sum _{\{k_{i}\}|\chi _{P}^{2}(\{k_{i}\},\{p_{i}\})>T}\prod _{i=1}^{m}\left({\frac {np_{i}}{k_{i}}}\right)^{k_{i}}{\sqrt {\frac {2\pi n}{\prod _{i=1}^{m}2\pi k_{i}}}}}$

By substituting for

${\displaystyle x_{i}={\frac {k_{i}-np_{i}}{\sqrt {n}}},\qquad i=1,\cdots ,m-1,}$

we may approximate for large ${\displaystyle n}$ the sum over the ${\displaystyle k_{i}}$ by an integral over the ${\displaystyle x_{i}}$. Noting that:

${\displaystyle k_{m}=np_{m}-{\sqrt {n}}\sum _{i=1}^{m-1}x_{i},}$

we arrive at

${\displaystyle {\begin{aligned}P(\chi _{P}^{2}(\{p_{i}\})>T)&\sim {\sqrt {\frac {2\pi n}{\prod _{i=1}^{m}2\pi k_{i}}}}\int _{\chi _{P}^{2}(\{{\sqrt {n}}x_{i}+np_{i}\},\{p_{i}\})>T}\left\{\prod _{i=1}^{m-1}{{\sqrt {n}}dx_{i}}\right\}\left\{\prod _{i=1}^{m-1}\left(1+{\frac {x_{i}}{{\sqrt {n}}p_{i}}}\right)^{-(np_{i}+{\sqrt {n}}x_{i})}\left(1-{\frac {\sum _{i=1}^{m-1}{x_{i}}}{{\sqrt {n}}p_{m}}}\right)^{-\left(np_{m}-{\sqrt {n}}\sum _{i=1}^{m-1}x_{i}\right)}\right\}\\&={\sqrt {\frac {2\pi n}{\prod _{i=1}^{m}\left(2\pi np_{i}+2\pi {\sqrt {n}}x_{i}\right)}}}\int _{\chi _{P}^{2}(\{{\sqrt {n}}x_{i}+np_{i}\},\{p_{i}\})>T}\left\{\prod _{i=1}^{m-1}{{\sqrt {n}}dx_{i}}\right\}\times \\&\qquad \qquad \times \left\{\prod _{i=1}^{m-1}\exp \left[-\left(np_{i}+{\sqrt {n}}x_{i}\right)\ln \left(1+{\frac {x_{i}}{{\sqrt {n}}p_{i}}}\right)\right]\exp \left[-\left(np_{m}-{\sqrt {n}}\sum _{i=1}^{m-1}x_{i}\right)\ln \left(1-{\frac {\sum _{i=1}^{m-1}{x_{i}}}{{\sqrt {n}}p_{m}}}\right)\right]\right\}\end{aligned}}}$

By expanding the logarithm and taking the leading terms in ${\displaystyle n}$, we get

${\displaystyle P(\chi _{P}^{2}(\{p_{i}\})>T)\sim {\frac {1}{\sqrt {(2\pi )^{m-1}\prod _{i=1}^{m}p_{i}}}}\int _{\chi _{P}^{2}(\{{\sqrt {n}}x_{i}+np_{i}\},\{p_{i}\})>T}\left\{\prod _{i=1}^{m-1}dx_{i}\right\}\prod _{i=1}^{m-1}\exp \left[-{\frac {1}{2}}\sum _{i=1}^{m-1}{\frac {x_{i}^{2}}{p_{i}}}-{\frac {1}{2p_{m}}}\left(\sum _{i=1}^{m-1}{x_{i}}\right)^{2}\right]}$

Pearson's chi, ${\displaystyle \chi _{P}^{2}(\{k_{i}\},\{p_{i}\})=\chi _{P}^{2}(\{{\sqrt {n}}x_{i}+np_{i}\},\{p_{i}\})}$, is precisely the argument of the exponent (except for the -1/2; note that the final term in the exponent's argument is equal to ${\displaystyle (k_{m}-np_{m})^{2}/(np_{m})}$).

This argument can be written as:

${\displaystyle -{\frac {1}{2}}\sum _{i,j=1}^{m-1}x_{i}A_{ij}x_{j},\qquad i,j=1,\cdots ,m-1,\quad A_{ij}={\tfrac {\delta _{ij}}{p_{i}}}+{\tfrac {1}{p_{m}}}.}$

${\displaystyle A}$ is a regular symmetric ${\displaystyle (m-1)\times (m-1)}$ matrix, and hence diagonalizable. It is therefore possible to make a linear change of variables in ${\displaystyle \{x_{i}\}}$ so as to get ${\displaystyle m-1}$ new variables ${\displaystyle \{y_{i}\}}$ so that:

${\displaystyle \sum _{i,j=1}^{m-1}x_{i}A_{ij}x_{j}=\sum _{i=1}^{m-1}y_{i}^{2}.}$

This linear change of variables merely multiplies the integral by a constant Jacobian, so we get:

${\displaystyle P(\chi _{P}^{2}(\{p_{i}\})>T)\sim C\int _{\sum _{i=1}^{m-1}y_{i}^{2}>T}\left\{\prod _{i=1}^{m-1}dy_{i}\right\}\prod _{i=1}^{m-1}\exp \left[-{\frac {1}{2}}\left(\sum _{i=1}^{m-1}y_{i}^{2}\right)\right]}$

Where C is a constant.

This is the probability that squared sum of ${\displaystyle m-1}$ independent normally distributed variables of zero mean and unit variance will be greater than T, namely that ${\displaystyle \chi ^{2}}$ with ${\displaystyle m-1}$ degrees of freedom is larger than T.

We have thus shown that at the limit where ${\displaystyle n\to \infty ,}$ the distribution of Pearson's chi approaches the chi distribution with ${\displaystyle m-1}$ degrees of freedom.