Likelihood-ratio test

Suppose that we have a statistical model with parameter space ${\displaystyle \Theta }$. A null hypothesis is often stated by saying that the parameter ${\displaystyle \theta }$ is in a specified subset ${\displaystyle \Theta _{0}}$ of ${\displaystyle \Theta }$. The alternative hypothesis is thus that ${\displaystyle \theta }$ is in the complement of ${\displaystyle \Theta _{0}}$, i.e. in ${\displaystyle \Theta ~\backslash ~\Theta _{0}}$, which is denoted by ${\displaystyle \Theta _{0}^{\text{c}}}$. The likelihood ratio test statistic for the null hypothesis ${\displaystyle H_{0}\,:\,\theta \in \Theta _{0}}$ is given by:[8]

${\displaystyle \lambda _{\text{LR}}=-2\ln \left[{\frac {~\sup _{\theta \in \Theta _{0}}{\mathcal {L}}(\theta )~}{~\sup _{\theta \in \Theta }{\mathcal {L}}(\theta )~}}\right]}$

where the quantity inside the brackets is called the likelihood ratio. Here, the ${\displaystyle \sup }$ notation refers to the supremum. As all likelihoods are positive, and as the constrained maximum cannot exceed the unconstrained maximum, the likelihood ratio is bounded between zero and one.

Often the likelihood-ratio test statistic is expressed as a difference between the log-likelihoods

${\displaystyle \lambda _{\text{LR}}=-2\left[~\ell (\theta _{0})-\ell ({\hat {\theta }})~\right]}$

where

${\displaystyle \ell ({\hat {\theta }})\equiv \ln \left[~\sup _{\theta \in \Theta }{\mathcal {L}}(\theta )~\right]~}$

is the logarithm of the maximized likelihood function ${\displaystyle {\mathcal {L}}}$, and ${\displaystyle \ell (\theta _{0})}$ is the maximal value in the special case that the null hypothesis is true (but not necessarily a value that maximizes ${\displaystyle {\mathcal {L}}}$ for the sampled data) and

${\displaystyle \theta _{0}\in \Theta _{0}\qquad {\text{ and }}\qquad {\hat {\theta }}\in \Theta ~}$

denote the respective arguments of the maxima and the allowed ranges they're embedded in. Multiplying by −2 ensures mathematically that (by Wilks' theorem) ${\displaystyle \lambda _{\text{LR}}}$ converges asymptotically to being χ²-distributed if the null hypothesis happens to be true.[9] The finite sample distributions of likelihood-ratio tests are generally unknown.[10]

The likelihood-ratio test requires that the models be nested – i.e. the more complex model can be transformed into the simpler model by imposing constraints on the former's parameters. Many common test statistics are tests for nested models and can be phrased as log-likelihood ratios or approximations thereof: e.g. the Z-test, the F-test, the G-test, and Pearson's chi-squared test; for an illustration with the one-sample t-test, see below.

If the models are not nested, then instead of the likelihood-ratio test, there is a generalization of the test that can usually be used: for details, see relative likelihood.

A simple-vs.-simple hypothesis test has completely specified models under both the null hypothesis and the alternative hypothesis, which for convenience are written in terms of fixed values of a notional parameter ${\displaystyle \theta }$:

${\displaystyle {\begin{aligned}H_{0}&:&\theta =\theta _{0},\\H_{1}&:&\theta =\theta _{1}.\end{aligned}}}$

In this case, under either hypothesis, the distribution of the data is fully specified: there are no unknown parameters to estimate. For this case, a variant of the likelihood-ratio test is available:[11][12]

${\displaystyle \Lambda (x)={\frac {~{\mathcal {L}}(\theta _{0}\mid x)~}{~{\mathcal {L}}(\theta _{1}\mid x)~}}}$

Some older references may use the reciprocal of the function above as the definition.[13] Thus, the likelihood ratio is small if the alternative model is better than the null model.

The likelihood-ratio test provides the decision rule as follows:

If ${\displaystyle ~\Lambda >c~}$, do not reject ${\displaystyle H_{0}}$;

If ${\displaystyle ~\Lambda <c~}$, reject ${\displaystyle H_{0}}$;

If ${\displaystyle ~\Lambda =c~}$, reject ${\displaystyle H_{0}}$ with probability ${\displaystyle ~q~}$.

The values ${\displaystyle c}$ and ${\displaystyle q}$ are usually chosen to obtain a specified significance level ${\displaystyle \alpha }$, via the relation

${\displaystyle ~q~}$ ${\displaystyle \operatorname {P} (\Lambda =c\mid H_{0})~+~\operatorname {P} (\Lambda <c\mid H_{0})~=~\alpha ~.}$

The Neyman–Pearson lemma states that this likelihood-ratio test is the most powerful among all level ${\displaystyle \alpha }$ tests for this case.[7][12]

The following example is adapted and abridged from Stuart, Ord & Arnold (1999, §22.2).

Suppose that we have a random sample, of size n, from a population that is normally-distributed. Both the mean, μ, and the standard deviation, σ, of the population are unknown. We want to test whether the mean is equal to a given value, μ₀ .

Thus, our null hypothesis is H₀:  μ = μ₀  and our alternative hypothesis is H₁:  μ ≠ μ₀ . The likelihood function is

${\displaystyle {\mathcal {L}}(\mu ,\sigma \mid x)=\left(2\pi \sigma ^{2}\right)^{-n/2}\exp \left(-\sum _{i=1}^{n}{\frac {(x_{i}-\mu )^{2}}{2\sigma ^{2}}}\right)\,.}$

With some calculation (omitted here), it can then be shown that

${\displaystyle \lambda =\left(1+{\frac {t^{2}}{n-1}}\right)^{-n/2}}$

where t is the t-statistic with n − 1 degrees of freedom. Hence we may use the known exact distribution of t_n−1 to draw inferences.