Lindley's paradox

The frequentist approach to testing ${\displaystyle \textstyle H_{0}}$ is to compute a p-value, the probability of observing a fraction of boys at least as large as ${\displaystyle \textstyle x}$ assuming ${\displaystyle \textstyle H_{0}}$ is true. Because the number of births is very large, we can use a normal approximation for the fraction of male births ${\displaystyle \textstyle X\sim N(\mu ,\sigma ^{2})}$, with ${\displaystyle \textstyle \mu =np=n\theta =98,451\times 0.5=49,225.5}$ and ${\displaystyle \textstyle \sigma ^{2}=n\theta (1-\theta )=98,451\times 0.5\times 0.5=24,612.75}$, to compute

${\displaystyle {\begin{aligned}P(X\geq x\mid \mu =49225.5)=\int _{x=49581}^{98451}{\frac {1}{\sqrt {2\pi \sigma ^{2}}}}e^{-({\frac {u-\mu }{\sigma }})^{2}/2}du\\=\int _{x=49581}^{98451}{\frac {1}{\sqrt {2\pi (24,612.75)}}}e^{-{\frac {(u-49225.5)^{2}}{24612.75}}/2}du\approx 0.0117.\end{aligned}}}$

We would have been equally surprised if we had seen 49,581 female births, i.e. ${\displaystyle \textstyle x\approx 0.4964}$, so a frequentist would usually perform a two-sided test, for which the p-value would be ${\displaystyle \textstyle p\approx 2\times 0.0117=0.0235}$. In both cases, the p-value is lower than the significance level, α, of 5%, so the frequentist approach rejects ${\displaystyle \textstyle H_{0}}$ as it disagrees with the observed data.

Assuming no reason to favor one hypothesis over the other, the Bayesian approach would be to assign prior probabilities ${\displaystyle \textstyle \pi (H_{0})=\pi (H_{1})=0.5}$ and a uniform distribution to ${\displaystyle \textstyle \theta }$ under ${\displaystyle H_{1}}$, and then to compute the posterior probability of ${\displaystyle \textstyle H_{0}}$ using Bayes' theorem,

${\displaystyle P(H_{0}\mid k)={\frac {P(k\mid H_{0})\pi (H_{0})}{P(k\mid H_{0})\pi (H_{0})+P(k\mid H_{1})\pi (H_{1})}}.}$

After observing ${\displaystyle \textstyle k=49,581}$ boys out of ${\displaystyle \textstyle n=98,451}$ births, we can compute the posterior probability of each hypothesis using the probability mass function for a binomial variable,

${\displaystyle {\begin{aligned}P(k\mid H_{0})&={n \choose k}(0.5)^{k}(1-0.5)^{n-k}\approx 1.95\times 10^{-4}\\P(k\mid H_{1})&=\int _{0}^{1}{n \choose k}\theta ^{k}(1-\theta )^{n-k}d\theta ={n \choose k}\mathrm {\mathrm {B} } (k+1,n-k+1)=1/(n+1)\approx 1.02\times 10^{-5}\end{aligned}}}$

where ${\displaystyle \textstyle \mathrm {\mathrm {B} } (a,b)}$ is the Beta function.

From these values, we find the posterior probability of ${\displaystyle P(\textstyle H_{0}\mid k)\approx 0.95}$, which strongly favors ${\displaystyle \textstyle H_{0}}$ over ${\displaystyle \textstyle H_{1}}$.

The two approaches—the Bayesian and the frequentist—appear to be in conflict, and this is the "paradox".

Naaman[3] proposed an adaption of the significance level to the sample size in order to control false positives: α_n, such that α_n = n^−r with r > 1/2. At least in the numerical example, taking r = 1/2, results in a significance level of 0.00318, so the frequentist would not reject the null hypothesis, which is in agreement with the Bayesian approach.

Distribution of p under the null hypothesis, and the posterior distribution of p.

If we use an uninformative prior and test a hypothesis more similar to that in the frequentist approach, the paradox disappears.

For example, if we calculate the posterior distribution ${\displaystyle \textstyle P(\theta \mid x,n)}$, using a uniform prior distribution on ${\displaystyle \textstyle \theta }$ (i.e. ${\displaystyle \textstyle \pi (\theta \in [0,1])=1}$), we find

${\displaystyle P(\theta \mid k,n)=\mathrm {\mathrm {B} } (k+1,n-k+1).}$

If we use this to check the probability that a newborn is more likely to be a boy than a girl, i.e. ${\displaystyle P(\theta >0.5\mid k,n)}$, we find

${\displaystyle \int _{0.5}^{1}\mathrm {\mathrm {B} } (49582,48871)\approx 0.983.}$

In other words, it is very likely that the proportion of male births is above 0.5.

Neither analysis gives an estimate of the effect size, directly, but both could be used to determine, for instance, if the fraction of boy births is likely to be above some particular threshold.

The apparent disagreement between the two approaches is caused by a combination of factors. First, the frequentist approach above tests ${\displaystyle \textstyle H_{0}}$ without reference to ${\displaystyle \textstyle H_{1}}$. The Bayesian approach evaluates ${\displaystyle \textstyle H_{0}}$ as an alternative to ${\displaystyle \textstyle H_{1}}$, and finds the first to be in better agreement with the observations. This is because the latter hypothesis is much more diffuse, as ${\displaystyle \textstyle \theta }$ can be anywhere in ${\displaystyle \textstyle [0,1]}$, which results in it having a very low posterior probability. To understand why, it is helpful to consider the two hypotheses as generators of the observations:

• Under ${\displaystyle \textstyle H_{0}}$, we choose ${\displaystyle \textstyle \theta \approx 0.500}$, and ask how likely it is to see 49,581 boys in 98,451 births.
• Under ${\displaystyle \textstyle H_{1}}$, we choose ${\displaystyle \textstyle \theta }$ randomly from anywhere within 0 to 1, and ask the same question.

Most of the possible values for ${\displaystyle \textstyle \theta }$ under ${\displaystyle \textstyle H_{1}}$ are very poorly supported by the observations. In essence, the apparent disagreement between the methods is not a disagreement at all, but rather two different statements about how the hypotheses relate to the data:

• The frequentist finds that ${\displaystyle \textstyle H_{0}}$ is a poor explanation for the observation.
• The Bayesian finds that ${\displaystyle \textstyle H_{0}}$ is a far better explanation for the observation than ${\displaystyle \textstyle H_{1}}$.

The ratio of the sex of newborns is improbably 50/50 male/female, according to the frequentist test. Yet 50/50 is a better approximation than most, but not all, other ratios. The hypothesis ${\displaystyle \textstyle \theta \approx 0.504}$ would have fit the observation much better than almost all other ratios, including ${\displaystyle \textstyle \theta \approx 0.500}$.

For example, this choice of hypotheses and prior probabilities implies the statement: "if ${\displaystyle \textstyle \theta }$ > 0.49 and ${\displaystyle \textstyle \theta }$ < 0.51, then the prior probability of ${\displaystyle \theta }$ being exactly 0.5 is 0.50/0.51 ${\displaystyle \approx }$ 98%." Given such a strong preference for ${\displaystyle \theta =0.5}$, it is easy to see why the Bayesian approach favors ${\displaystyle H_{0}}$ in the face of ${\displaystyle x\approx 0.5036}$, even though the observed value of ${\displaystyle x}$ lies ${\displaystyle 2.28\sigma }$ away from 0.5. The deviation of over 2 sigma from ${\displaystyle H_{0}}$ is considered significant in the frequentist approach, but its significance is overruled by the prior in the Bayesian approach.

Looking at it another way, we can see that the prior distribution is essentially flat with a delta function at ${\displaystyle \textstyle \theta =0.5}$. Clearly this is dubious. In fact if you were to picture real numbers as being continuous, then it would be more logical to assume that it would impossible for any given number to be exactly the parameter value, i.e., we should assume ${\displaystyle P(\theta =0.5)=0}$.

A more realistic distribution for ${\displaystyle \textstyle \theta }$ in the alternative hypothesis produces a less surprising result for the posterior of ${\displaystyle \textstyle H_{0}}$. For example, if we replace ${\displaystyle \textstyle H_{1}}$ with ${\displaystyle \textstyle H_{2}:\theta =x}$, i.e., the maximum likelihood estimate for ${\displaystyle \textstyle \theta }$, the posterior probability of ${\displaystyle \textstyle H_{0}}$ would be only 0.07 compared to 0.93 for ${\displaystyle \textstyle H_{2}}$ (Of course, one cannot actually use the MLE as part of a prior distribution).