Working–Hotelling procedure

Consider a simple linear regression model ${\displaystyle Y=\beta _{0}+\beta _{1}X+\varepsilon }$, where ${\displaystyle Y}$ is the response variable and ${\displaystyle X}$ the explanatory variable, and let ${\displaystyle b_{0}}$ and ${\displaystyle b_{1}}$ be the least-squares estimates of ${\displaystyle \beta _{0}}$ and ${\displaystyle \beta _{1}}$ respectively. Then the least-squares estimate of the mean response ${\displaystyle E(Y_{i})}$ at the level ${\displaystyle X=x_{i}}$ is ${\displaystyle {\hat {Y_{i}}}=b_{0}+b_{1}x_{i}}$. It can then be shown, assuming that the errors independently and identically follow the normal distribution, that an ${\displaystyle 1-\alpha }$ confidence interval of the mean response at a certain level of ${\displaystyle X}$ is as follows:

${\displaystyle {\hat {y}}_{i}\in \left[b_{0}+b_{1}x_{i}\pm t_{\alpha /2,{\text{df}}=n-2}{\sqrt {\left({\frac {1}{n-2}}\sum _{j=1}^{n}e_{j}^{\,2}\right)\cdot \left({\frac {1}{n}}+{\frac {(x_{i}-{\bar {x}})^{2}}{\sum _{j=1}^{n}(x_{j}-{\bar {x}})^{2}}}\right)}}\right],}$

where ${\displaystyle \left({\frac {1}{n-2}}\sum _{j=1}^{n}e_{j}^{\,2}\right)}$ is the mean squared error and ${\displaystyle t_{\alpha /2,{\text{df}}=n-2}}$ denotes the upper ${\displaystyle {\frac {\alpha }{2}}^{\text{th}}}$ percentile of Student's t-distribution with ${\displaystyle n-2}$ degrees of freedom.

However, as multiple mean responses are estimated, the confidence level declines rapidly. To fix the confidence coefficient at ${\displaystyle 1-\alpha }$, the Working–Hotelling approach employs an F-statistic:[2][3]

${\displaystyle {\hat {y}}_{i}\in \left[b_{0}+b_{1}x_{i}\pm W{\sqrt {\left({\frac {1}{n-2}}\sum _{j=1}^{n}e_{j}^{\,2}\right)\cdot \left({\frac {1}{n}}+{\frac {(x_{i}-{\bar {x}})^{2}}{\sum _{j=1}^{n}(x_{j}-{\bar {x}})^{2}}}\right)}}\right],}$

where ${\displaystyle W^{2}=2F_{\alpha ,{\text{df}}=(2,n-2)}}$ and ${\displaystyle F}$ denotes the upper ${\displaystyle \alpha ^{\text{th}}}$ percentile of the F-distribution with ${\displaystyle (2,n-2)}$ degrees of freedom. The confidence level of is ${\displaystyle 1-\alpha }$ over all values of ${\displaystyle X}$, i.e. ${\displaystyle x_{i}\in \mathbb {R} }$.

The Working–Hotelling confidence bands can be easily generalised to multiple linear regression. Consider a general linear model as defined in the linear regressions article, that is,

${\displaystyle \mathbf {Y} =\mathbf {X} {\boldsymbol {\beta }}+{\boldsymbol {\varepsilon }},\,}$

where

${\displaystyle \mathbf {Y} ={\begin{pmatrix}Y_{1}\\Y_{2}\\\vdots \\Y_{n}\end{pmatrix}},\quad \mathbf {X} ={\begin{pmatrix}\mathbf {x} _{1}^{\rm {T}}\\\mathbf {x} _{2}^{\rm {T}}\\\vdots \\\mathbf {x} _{n}^{\rm {T}}\end{pmatrix}}={\begin{pmatrix}x_{11}&\cdots &x_{1p}\\x_{21}&\cdots &x_{2p}\\\vdots &\ddots &\vdots \\x_{n1}&\cdots &x_{np}\end{pmatrix}},{\boldsymbol {\beta }}={\begin{pmatrix}\beta _{1}\\\beta _{2}\\\vdots \\\beta _{p}\end{pmatrix}},\quad {\boldsymbol {\varepsilon }}={\begin{pmatrix}\varepsilon _{1}\\\varepsilon _{2}\\\vdots \\\varepsilon _{n}\end{pmatrix}}.}$

Again, it can be shown that the least-squares estimate of the mean response ${\displaystyle E(Y_{i})=\mathbf {x} _{i}^{\rm {T}}{\boldsymbol {\beta }}}$ is ${\displaystyle {\hat {Y}}_{i}=\mathbf {x} _{i}^{\rm {T}}\mathbf {b} }$, where ${\displaystyle \mathbf {b} }$ consists of least-square estimates of the entries in ${\displaystyle {\boldsymbol {\beta }}}$, i.e. ${\displaystyle \mathbf {b} =(\mathbf {X} ^{\rm {T}}\mathbf {X} )^{-1}\mathbf {X} ^{\rm {T}}\mathbf {Y} }$. Likewise, it can be shown that a ${\displaystyle 1-\alpha }$ confidence interval for a single mean response estimate is as follows:[4]

${\displaystyle {\hat {y}}_{i}\in \left[\mathbf {x} _{i}^{\rm {T}}\mathbf {b} \pm t_{\alpha /2,{\text{df}}=n-p}{\sqrt {\operatorname {MSE} (\mathbf {x} _{i}^{\rm {T}}(\mathbf {X} ^{\rm {T}}\mathbf {X} )^{-1}\mathbf {x} _{i}}})\right],}$

where ${\displaystyle \operatorname {MSE} }$ is the observed value of the mean squared error ${\displaystyle (Y^{\rm {T}}Y-\mathbf {b} ^{\rm {T}}X^{\rm {T}}Y)}$.

The Working–Hotelling approach to multiple estimations is similar to that of simple linear regression, with only a change in the degrees of freedom:[3]

${\displaystyle {\hat {y}}_{i}\in \left[\mathbf {x} _{i}^{\rm {T}}\mathbf {b} \pm W{\sqrt {\operatorname {MSE} (\mathbf {x} _{i}^{\rm {T}}(\mathbf {X} ^{\rm {T}}\mathbf {X} )^{-1}\mathbf {x} _{i}}})\right],}$

where ${\displaystyle W^{2}=2F_{\alpha ,{\text{df}}=(p,n-p)}}$.