Omitted-variable bias

Suppose the true cause-and-effect relationship is given by:

${\displaystyle y=a+bx+cz+u}$

with parameters a, b, c, dependent variable y, independent variables x and z, and error term u. We wish to know the effect of x itself upon y (that is, we wish to obtain an estimate of b).

Two conditions must hold true for omitted-variable bias to exist in linear regression:

• the omitted variable must be a determinant of the dependent variable (i.e., its true regression coefficient must not be zero); and
• the omitted variable must be correlated with an independent variable specified in the regression (i.e., cov(z,x) must not equal zero).

Suppose we omit z from the regression, and suppose the relation between x and z is given by

${\displaystyle z=d+fx+e}$

with parameters d, f and error term e. Substituting the second equation into the first gives

${\displaystyle y=(a+cd)+(b+cf)x+(u+ce).}$

If a regression of y is conducted upon x only, this last equation is what is estimated, and the regression coefficient on x is actually an estimate of (b + cf ), giving not simply an estimate of the desired direct effect of x upon y (which is b), but rather of its sum with the indirect effect (the effect f of x on z times the effect c of z on y). Thus by omitting the variable z from the regression, we have estimated the total derivative of y with respect to x rather than its partial derivative with respect to x. These differ if both c and f are non-zero.

The direction and extent of the bias are both contained in cf, since the effect sought is b but the regression estimates b+cf. The extent of the bias is the absolute value of cf, and the direction of bias is upward (toward a more positive or less negative value) if cf > 0 (if the direction of correlation between y and z is the same as that between x and z), and it is downward otherwise.

As an example, consider a linear model of the form

${\displaystyle y_{i}=x_{i}\beta +z_{i}\delta +u_{i},\qquad i=1,\dots ,n}$

where

• x_i is a 1 × p row vector of values of p independent variables observed at time i or for the i ^th study participant;
• β is a p × 1 column vector of unobservable parameters (the response coefficients of the dependent variable to each of the p independent variables in x_i) to be estimated;
• z_i is a scalar and is the value of another independent variable that is observed at time i or for the i ^th study participant;
• δ is a scalar and is an unobservable parameter (the response coefficient of the dependent variable to z_i) to be estimated;
• u_i is the unobservable error term occurring at time i or for the i ^th study participant; it is an unobserved realization of a random variable having expected value 0 (conditionally on x_i and z_i);
• y_i is the observation of the dependent variable at time i or for the i ^th study participant.

We collect the observations of all variables subscripted i = 1, ..., n, and stack them one below another, to obtain the matrix X and the vectors Y, Z, and U:

${\displaystyle X=\left[{\begin{array}{c}x_{1}\\\vdots \\x_{n}\end{array}}\right]\in \mathbb {R} ^{n\times p},}$

and

${\displaystyle Y=\left[{\begin{array}{c}y_{1}\\\vdots \\y_{n}\end{array}}\right],\quad Z=\left[{\begin{array}{c}z_{1}\\\vdots \\z_{n}\end{array}}\right],\quad U=\left[{\begin{array}{c}u_{1}\\\vdots \\u_{n}\end{array}}\right]\in \mathbb {R} ^{n\times 1}.}$

If the independent variable z is omitted from the regression, then the estimated values of the response parameters of the other independent variables will be given by the usual least squares calculation,

${\displaystyle {\widehat {\beta }}=(X'X)^{-1}X'Y\,}$

(where the "prime" notation means the transpose of a matrix and the -1 superscript is matrix inversion).

Substituting for Y based on the assumed linear model,

${\displaystyle {\begin{aligned}{\widehat {\beta }}&=(X'X)^{-1}X'(X\beta +Z\delta +U)\\&=(X'X)^{-1}X'X\beta +(X'X)^{-1}X'Z\delta +(X'X)^{-1}X'U\\&=\beta +(X'X)^{-1}X'Z\delta +(X'X)^{-1}X'U.\end{aligned}}}$

On taking expectations, the contribution of the final term is zero; this follows from the assumption that U is uncorrelated with the regressors X. On simplifying the remaining terms:

${\displaystyle {\begin{aligned}E[{\widehat {\beta }}\mid X]&=\beta +(X'X)^{-1}E[X'Z\mid X]\delta \\&=\beta +{\text{bias}}.\end{aligned}}}$

The second term after the equal sign is the omitted-variable bias in this case, which is non-zero if the omitted variable z is correlated with any of the included variables in the matrix X (that is, if X′Z does not equal a vector of zeroes). Note that the bias is equal to the weighted portion of z_i which is "explained" by x_i.