Sensitivity index

For two univariate distributions ${\displaystyle a}$ and ${\displaystyle b}$ with the same standard deviation, it is denoted by ${\displaystyle d'}$ ('dee-prime'):

${\displaystyle d'={\frac {\left\vert \mu _{a}-\mu _{b}\right\vert }{\sigma }}}$.

In higher dimensions, i.e. with two multivariate distributions with the same variance-covariance matrix ${\displaystyle \mathbf {\Sigma } }$, (whose symmetric square-root, the standard deviation matrix, is ${\displaystyle \mathbf {S} }$), this generalizes to the Mahalanobis distance between the two distributions:

${\displaystyle d'={\sqrt {({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})'\mathbf {\Sigma } ^{-1}({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})}}=\lVert \mathbf {S} ^{-1}({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})\rVert =\lVert {\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b}\rVert /\sigma _{\boldsymbol {\mu }}}$,

where ${\displaystyle \sigma _{\boldsymbol {\mu }}=1/\lVert \mathbf {S} ^{-1}{\boldsymbol {\mu }}\rVert }$ is the 1d slice of the sd along the unit vector ${\displaystyle {\boldsymbol {\mu }}}$ through the means, i.e. the ${\displaystyle d'}$ equals the ${\displaystyle d'}$ along the 1d slice through the means.[1]

For two bivariate distributions with equal variance-covariance, this is given by:

${\displaystyle {d'}^{2}={\frac {1}{1-\rho ^{2}}}\left({d'}_{x}^{2}+{d'}_{y}^{2}-2\rho {d'}_{x}{d'}_{y}\right)}$,

where ${\displaystyle \rho }$ is the correlation coefficient, and here ${\displaystyle d'_{x}={\frac {{\mu _{b}}_{x}-{\mu _{a}}_{x}}{\sigma _{x}}}}$ and ${\displaystyle d'_{y}={\frac {{\mu _{b}}_{y}-{\mu _{a}}_{y}}{\sigma _{y}}}}$, i.e. including the signs of the mean differences instead of the absolute.[1]

${\displaystyle d'}$ is also estimated as ${\displaystyle Z({\text{hit rate}})-Z({\text{false alarm rate}})}$ [2].^: 7

When the two distributions have different standard deviations (or in general dimensions, different covariance matrices), there exist several contending indices, all of which reduce to ${\displaystyle d'}$ for equal variance/covariance.

This is the maximum (Bayes-optimal) discriminability index for two distributions, based on the amount of their overlap, i.e. the optimal (Bayes) error of classification ${\displaystyle e_{b}}$ by an ideal observer, or its complement, the optimal accuracy ${\displaystyle a_{b}}$:

${\displaystyle d'_{b}=-2Z\left({\text{Bayes error rate }}e_{b}\right)=2Z\left({\text{best accuracy rate }}a_{b}\right)}$,[1]

where ${\displaystyle Z}$ is the inverse cumulative distribution function of the standard normal. The Bayes discriminability between univariate or multivariate normal distributions can be numerically computed [1] (Matlab code), and may also be used as an approximation when the distributions are close to normal.

${\displaystyle d'_{b}}$ is a positive-definite statistical distance measure that is free of assumptions about the distributions, like the Kullback-Leibler divergence ${\displaystyle D_{\text{KL}}}$. ${\displaystyle D_{\text{KL}}(a,b)}$ is asymmetric, whereas ${\displaystyle d'_{b}(a,b)}$ is symmetric for the two distributions. However, ${\displaystyle d'_{b}}$ does not satisfy the triangle inequality, so it is not a full metric. [1]

In particular, for a yes/no task between two univariate normal distributions with means ${\displaystyle \mu _{a},\mu _{b}}$ and variances ${\displaystyle v_{a}>v_{b}}$, the Bayes-optimal classification accuracies are:[1]

${\displaystyle p(A|a)=p({\chi '}_{1,v_{a}\lambda }^{2}>v_{b}c),\;\;p(B|b)=p({\chi '}_{1,v_{b}\lambda }^{2}<v_{a}c)}$,

where ${\displaystyle \chi '^{2}}$ denotes the non-central chi-squared distribution, ${\displaystyle \lambda =\left({\frac {\mu _{a}-\mu _{b}}{v_{a}-v_{b}}}\right)^{2}}$, and ${\displaystyle c=\lambda +{\frac {\ln v_{a}-\ln v_{b}}{v_{a}-v_{b}}}}$. The Bayes discriminability ${\displaystyle d'_{b}=2Z\left({\frac {p\left(A|a\right)+p\left(B|b\right)}{2}}\right).}$

${\displaystyle d'_{b}}$ can also be computed from the ROC curve of a yes/no task between two univariate normal distributions with a single shifting criterion. It can also be computed from the ROC curve of any two distributions (in any number of variables) with a shifting likelihood-ratio, by locating the point on the ROC curve that is farthest from the diagonal. [1]

For a two-interval task between these distributions, the optimal accuracy is ${\displaystyle a_{b}=p\left({\tilde {\chi }}_{{\boldsymbol {w}},{\boldsymbol {k}},{\boldsymbol {\lambda }},0,0}^{2}>0\right)}$ (${\displaystyle {\tilde {\chi }}^{2}}$ denotes the generalized chi-squared distribution), where ${\displaystyle {\boldsymbol {w}}={\begin{bmatrix}\sigma _{s}^{2}&-\sigma _{n}^{2}\end{bmatrix}},\;{\boldsymbol {k}}={\begin{bmatrix}1&1\end{bmatrix}},\;{\boldsymbol {\lambda }}={\frac {\mu _{s}-\mu _{n}}{\sigma _{s}^{2}-\sigma _{n}^{2}}}{\begin{bmatrix}\sigma _{s}^{2}&\sigma _{n}^{2}\end{bmatrix}}}$.[1] The Bayes discriminability ${\displaystyle d'_{b}=2Z\left(a_{b}\right)}$.

A common approximate (i.e. sub-optimal) discriminability index that has a closed-form is to take the average of the variances, i.e. the rms of the two standard deviations: ${\displaystyle d'_{a}=\left\vert \mu _{a}-\mu _{b}\right\vert /\sigma _{\text{rms}}}$ [3] (also denoted by ${\displaystyle d_{a}}$). It is ${\displaystyle {\sqrt {2}}}$ times the ${\displaystyle z}$-score of the area under the receiver operating characteristic curve (AUC) of a single-criterion observer. This index is extended to general dimensions as the Mahalanobis distance using the pooled covariance, i.e. with ${\displaystyle \mathbf {S} _{\text{rms}}=\left[\left(\mathbf {\Sigma } _{a}+\mathbf {\Sigma } _{b}\right)/2\right]^{\frac {1}{2}}}$ as the common sd matrix.[1]

Another index is ${\displaystyle d'_{e}=\left\vert \mu _{a}-\mu _{b}\right\vert /\sigma _{\text{avg}}}$, extended to general dimensions using ${\displaystyle \mathbf {S} _{\text{avg}}=\left(\mathbf {S} _{a}+\mathbf {S} _{b}\right)/2}$ as the common sd matrix.[1]