Regression dilution

The case that x is fixed, but measured with noise, is known as the functional model or functional relationship.[2] It can be corrected using total least squares[3] and errors-in-variables models in general.

The case that the x variable arises randomly is known as the structural model or structural relationship. For example, in a medical study patients are recruited as a sample from a population, and their characteristics such as blood pressure may be viewed as arising from a random sample.

Under certain assumptions (typically, normal distribution assumptions) there is a known ratio between the true slope, and the expected estimated slope. Frost and Thompson (2000) review several methods for estimating this ratio and hence correcting the estimated slope.[4] The term regression dilution ratio, although not defined in quite the same way by all authors, is used for this general approach, in which the usual linear regression is fitted, and then a correction applied. The reply to Frost & Thompson by Longford (2001) refers the reader to other methods, expanding the regression model to acknowledge the variability in the x variable, so that no bias arises.[5] Fuller (1987) is one of the standard references for assessing and correcting for regression dilution.[6]

Hughes (1993) shows that the regression dilution ratio methods apply approximately in survival models.[7] Rosner (1992) shows that the ratio methods apply approximately to logistic regression models.[8] Carroll et al. (1995) give more detail on regression dilution in nonlinear models, presenting the regression dilution ratio methods as the simplest case of regression calibration methods, in which additional covariates may also be incorporated.[9]

In general, methods for the structural model require some estimate of the variability of the x variable. This will require repeated measurements of the x variable in the same individuals, either in a sub-study of the main data set, or in a separate data set. Without this information it will not be possible to make a correction.

The case of multiple predictor variables subject to variability (possibly correlated) has been well-studied for linear regression, and for some non-linear regression models.[6][9] Other non-linear models, such as proportional hazards models for survival analysis, have been considered only with a single predictor subject to variability.[7]

Let ${\displaystyle \beta }$ and ${\displaystyle \theta }$ be the true values of two attributes of some person or statistical unit. These values are variables by virtue of the assumption that they differ for different statistical units in the population. Let ${\displaystyle {\hat {\beta }}}$ and ${\displaystyle {\hat {\theta }}}$ be estimates of ${\displaystyle \beta }$ and ${\displaystyle \theta }$ derived either directly by observation-with-error or from application of a measurement model, such as the Rasch model. Also, let

${\displaystyle {\hat {\beta }}=\beta +\epsilon _{\beta },\quad \quad {\hat {\theta }}=\theta +\epsilon _{\theta },}$

where ${\displaystyle \epsilon _{\beta }}$ and ${\displaystyle \epsilon _{\theta }}$ are the measurement errors associated with the estimates ${\displaystyle {\hat {\beta }}}$ and ${\displaystyle {\hat {\theta }}}$.

The estimated correlation between two sets of estimates is

${\displaystyle \operatorname {corr} ({\hat {\beta }},{\hat {\theta }})={\frac {\operatorname {cov} ({\hat {\beta }},{\hat {\theta }})}{{\sqrt {\operatorname {var} [{\hat {\beta }}]\operatorname {var} [{\hat {\theta }}}}]}}}$

${\displaystyle ={\frac {\operatorname {cov} (\beta +\epsilon _{\beta },\theta +\epsilon _{\theta })}{\sqrt {\operatorname {var} [\beta +\epsilon _{\beta }]\operatorname {var} [\theta +\epsilon _{\theta }]}}},}$

which, assuming the errors are uncorrelated with each other and with the true attribute values, gives

${\displaystyle \operatorname {corr} ({\hat {\beta }},{\hat {\theta }})={\frac {\operatorname {cov} (\beta ,\theta )}{\sqrt {(\operatorname {var} [\beta ]+\operatorname {var} [\epsilon _{\beta }])(\operatorname {var} [\theta ]+\operatorname {var} [\epsilon _{\theta }])}}}}$

${\displaystyle ={\frac {\operatorname {cov} (\beta ,\theta )}{\sqrt {(\operatorname {var} [\beta ]\operatorname {var} [\theta ])}}}.{\frac {\sqrt {\operatorname {var} [\beta ]\operatorname {var} [\theta ]}}{\sqrt {(\operatorname {var} [\beta ]+\operatorname {var} [\epsilon _{\beta }])(\operatorname {var} [\theta ]+\operatorname {var} [\epsilon _{\theta }])}}}}$

${\displaystyle =\rho {\sqrt {R_{\beta }R_{\theta }}},}$

where ${\displaystyle R_{\beta }}$ is the separation index of the set of estimates of ${\displaystyle \beta }$, which is analogous to Cronbach's alpha; that is, in terms of classical test theory, ${\displaystyle R_{\beta }}$ is analogous to a reliability coefficient. Specifically, the separation index is given as follows:

${\displaystyle R_{\beta }={\frac {\operatorname {var} [\beta ]}{\operatorname {var} [\beta ]+\operatorname {var} [\epsilon _{\beta }]}}={\frac {\operatorname {var} [{\hat {\beta }}]-\operatorname {var} [\epsilon _{\beta }]}{\operatorname {var} [{\hat {\beta }}]}},}$

where the mean squared standard error of person estimate gives an estimate of the variance of the errors, ${\displaystyle \epsilon _{\beta }}$. The standard errors are normally produced as a by-product of the estimation process (see Rasch model estimation).

The disattenuated estimate of the correlation between the two sets of parameter estimates is therefore

${\displaystyle \rho ={\frac {{\mbox{corr}}({\hat {\beta }},{\hat {\theta }})}{\sqrt {R_{\beta }R_{\theta }}}}.}$

That is, the disattenuated correlation estimate is obtained by dividing the correlation between the estimates by the geometric mean of the separation indices of the two sets of estimates. Expressed in terms of classical test theory, the correlation is divided by the geometric mean of the reliability coefficients of two tests.

Given two random variables ${\displaystyle X^{\prime }}$ and ${\displaystyle Y^{\prime }}$ measured as ${\displaystyle X}$ and ${\displaystyle Y}$ with measured correlation ${\displaystyle r_{xy}}$ and a known reliability for each variable, ${\displaystyle r_{xx}}$ and ${\displaystyle r_{yy}}$, the estimated correlation between ${\displaystyle X^{\prime }}$ and ${\displaystyle Y^{\prime }}$ corrected for attenuation is

${\displaystyle r_{x'y'}={\frac {r_{xy}}{\sqrt {r_{xx}r_{yy}}}}}$.

How well the variables are measured affects the correlation of X and Y. The correction for attenuation tells one what the estimated correlation is expected to be if one could measure X′ and Y′ with perfect reliability.

Thus if ${\displaystyle X}$ and ${\displaystyle Y}$ are taken to be imperfect measurements of underlying variables ${\displaystyle X'}$ and ${\displaystyle Y'}$ with independent errors, then ${\displaystyle r_{x'y'}}$ estimates the true correlation between ${\displaystyle X'}$ and ${\displaystyle Y'}$.