Fay–Herriot model

In classical Fay–Herriot (FH), the data used for estimation are aggregate estimates for the subgroups based on surveys.

The model can also be applied to microdata. Consider rows of observations numbered j=1 to J, in groups from i=1 to I, with predictive data ${\displaystyle X_{ij}}$ for dependent variable ${\displaystyle Y_{ij}}$. If the model includes random effects only, it can be expressed by:

${\displaystyle Y_{ij}=\mu +\beta X_{ij}+U_{i}+\epsilon _{ij}}$

A probability distribution is assumed for the random effects ${\displaystyle U_{i}}$, typically a normal distribution. A different distribution can be assumed, e.g. if the sample distribution is known to have heavy tails.[1]

Often fixed effects are included, making it a mixed model, with auxiliary data and economic or probability assumptions that make it possible to identify these effects separately from one another and from sampling variation ${\displaystyle \epsilon _{ij}}$.[2]

The parameters of interest including the random effects are estimated together iteratively. Methods can include maximum likelihood estimation, the method of moments, or a Bayesian way.[3][4][5]

Fay–Herriot models can be characterized either as mixed models, or in a hierarchical form,[6] or a multilevel regression with poststratification.[7][8][9][10]

The resulting estimates for each area (subgroup) are weighted averages from the direct estimates and indirect estimates based on estimates of variances.

For random effects models to make consistent estimates, it is necessary that the subgroup-specific effects be uncorrelated to the other predictor variables in the model. If the subgroup-specific effects are correlated, then random effects estimation would be biased but fixed effects estimation would not be biased.

That correlation can be tested by running both the fixed effects and the random effects models and then applying the Hausman specification test. The test may not reject the hypothesis of no-correlation even when it is false, a Type II error, so that it cannot be definitively concluded that random effects estimation is unbiased even if the Hausman test fails to reject.

Robert Fay and Roger Herriot of the U.S. Census Bureau developed the model to make estimates for populations in each of many geographic regions. The authors referred to the method as a James–Stein procedure and did not use the term "random effects."[11] It is an area-level model.[12] The model has been used for the same purpose, called small-area estimation, by other U.S. government agencies.[6][13]

Rao and Molina's small area estimation text is sometimes characterized of as a definitive source about the FH model.[14]

The FH model is used extensively in the Small Area Income and Poverty Estimates (SAIPE) program of the U.S. Census Bureau.[15]