Distance correlation

Let us start with the definition of the sample distance covariance. Let (X_k, Y_k), k = 1, 2, ..., n be a statistical sample from a pair of real valued or vector valued random variables (X, Y). First, compute the n by n distance matrices (a_{j, k}) and (b_{j, k}) containing all pairwise distances

${\displaystyle {\begin{aligned}a_{j,k}&=\|X_{j}-X_{k}\|,\qquad j,k=1,2,\ldots ,n,\\b_{j,k}&=\|Y_{j}-Y_{k}\|,\qquad j,k=1,2,\ldots ,n,\end{aligned}}}$

where ||⋅ ||denotes Euclidean norm. Then take all doubly centered distances

${\displaystyle A_{j,k}:=a_{j,k}-{\overline {a}}_{j\cdot }-{\overline {a}}_{\cdot k}+{\overline {a}}_{\cdot \cdot },\qquad B_{j,k}:=b_{j,k}-{\overline {b}}_{j\cdot }-{\overline {b}}_{\cdot k}+{\overline {b}}_{\cdot \cdot },}$

where ${\displaystyle \textstyle {\overline {a}}_{j\cdot }}$ is the j-th row mean, ${\displaystyle \textstyle {\overline {a}}_{\cdot k}}$ is the k-th column mean, and ${\displaystyle \textstyle {\overline {a}}_{\cdot \cdot }}$ is the grand mean of the distance matrix of the X sample. The notation is similar for the b values. (In the matrices of centered distances (A_{j, k}) and (B_j,k) all rows and all columns sum to zero.) The squared sample distance covariance (a scalar) is simply the arithmetic average of the products A_{j, k} B_{j, k}:

${\displaystyle \operatorname {dCov} _{n}^{2}(X,Y):={\frac {1}{n^{2}}}\sum _{j=1}^{n}\sum _{k=1}^{n}A_{j,k}\,B_{j,k}.}$

The statistic T_n = n dCov²_n(X, Y) determines a consistent multivariate test of independence of random vectors in arbitrary dimensions. For an implementation see dcov.test function in the energy package for R.[4]

The population value of distance covariance can be defined along the same lines. Let X be a random variable that takes values in a p-dimensional Euclidean space with probability distribution μ and let Y be a random variable that takes values in a q-dimensional Euclidean space with probability distribution ν, and suppose that X and Y have finite expectations. Write

${\displaystyle a_{\mu }(x):=\operatorname {E} [\|X-x\|],\quad D(\mu ):=\operatorname {E} [a_{\mu }(X)],\quad d_{\mu }(x,x'):=\|x-x'\|-a_{\mu }(x)-a_{\mu }(x')+D(\mu ).}$

Finally, define the population value of squared distance covariance of X and Y as

${\displaystyle \operatorname {dCov} ^{2}(X,Y):=\operatorname {E} {\big [}d_{\mu }(X,X')d_{\nu }(Y,Y'){\big ]}.}$

One can show that this is equivalent to the following definition:

${\displaystyle {\begin{aligned}\operatorname {dCov} ^{2}(X,Y):={}&\operatorname {E} [\|X-X'\|\,\|Y-Y'\|]+\operatorname {E} [\|X-X'\|]\,\operatorname {E} [\|Y-Y'\|]\\&\qquad {}-\operatorname {E} [\|X-X'\|\,\|Y-Y''\|]-\operatorname {E} [\|X-X''\|\,\|Y-Y'\|]\\={}&\operatorname {E} [\|X-X'\|\,\|Y-Y'\|]+\operatorname {E} [\|X-X'\|]\,\operatorname {E} [\|Y-Y'\|]\\&\qquad {}-2\operatorname {E} [\|X-X'\|\,\|Y-Y''\|],\end{aligned}}}$

where E denotes expected value, and ${\displaystyle \textstyle (X,Y),}$ ${\displaystyle \textstyle (X',Y'),}$ and ${\displaystyle \textstyle (X'',Y'')}$ are independent and identically distributed. The primed random variables ${\displaystyle \textstyle (X',Y')}$ and ${\displaystyle \textstyle (X'',Y'')}$ denote independent and identically distributed (iid) copies of the variables ${\displaystyle X}$ and ${\displaystyle Y}$ and are similarly iid.[5] Distance covariance can be expressed in terms of the classical Pearson's covariance, cov, as follows:

${\displaystyle \operatorname {dCov} ^{2}(X,Y)=\operatorname {cov} (\|X-X'\|,\|Y-Y'\|)-2\operatorname {cov} (\|X-X'\|,\|Y-Y''\|).}$

This identity shows that the distance covariance is not the same as the covariance of distances, cov(‖X − X' ‖, ‖Y − Y' ‖). This can be zero even if X and Y are not independent.

Alternatively, the distance covariance can be defined as the weighted L² norm of the distance between the joint characteristic function of the random variables and the product of their marginal characteristic functions:[6]

${\displaystyle \operatorname {dCov} ^{2}(X,Y)={\frac {1}{c_{p}c_{q}}}\int _{\mathbb {R} ^{p+q}}{\frac {\left|\varphi _{X,Y}(s,t)-\varphi _{X}(s)\varphi _{Y}(t)\right|^{2}}{|s|_{p}^{1+p}|t|_{q}^{1+q}}}\,dt\,ds}$

where ${\displaystyle \varphi _{X,Y}(s,t)}$, ${\displaystyle \varphi _{X}(s)}$, and ${\displaystyle \varphi _{Y}(t)}$ are the characteristic functions of (X, Y), X, and Y, respectively, p, q denote the Euclidean dimension of X and Y, and thus of s and t, and c_p, c_q are constants. The weight function ${\displaystyle ({c_{p}c_{q}}{|s|_{p}^{1+p}|t|_{q}^{1+q}})^{-1}}$ is chosen to produce a scale equivariant and rotation invariant measure that doesn't go to zero for dependent variables.[6][7] One interpretation of the characteristic function definition is that the variables e^isX and e^itY are cyclic representations of X and Y with different periods given by s and t, and the expression ϕ_{X, Y}(s, t) − ϕ_X(s) ϕ_Y(t) in the numerator of the characteristic function definition of distance covariance is simply the classical covariance of e^isX and e^itY. The characteristic function definition clearly shows that dCov²(X, Y) = 0 if and only if X and Y are independent.

The distance variance is a special case of distance covariance when the two variables are identical. The population value of distance variance is the square root of

${\displaystyle \operatorname {dVar} ^{2}(X):=\operatorname {E} [\|X-X'\|^{2}]+\operatorname {E} ^{2}[\|X-X'\|]-2\operatorname {E} [\|X-X'\|\,\|X-X''\|],}$

where ${\displaystyle X}$, ${\displaystyle X'}$, and ${\displaystyle X''}$ are independent and identically distributed random variables, ${\displaystyle \operatorname {E} }$ denotes the expected value, and ${\displaystyle f^{2}(\cdot )=(f(\cdot ))^{2}}$ for function ${\displaystyle f(\cdot )}$, e.g., ${\displaystyle \operatorname {E} ^{2}[\cdot ]=(\operatorname {E} [\cdot ])^{2}}$.

The sample distance variance is the square root of

${\displaystyle \operatorname {dVar} _{n}^{2}(X):=\operatorname {dCov} _{n}^{2}(X,X)={\tfrac {1}{n^{2}}}\sum _{k,\ell }A_{k,\ell }^{2},}$

which is a relative of Corrado Gini's mean difference introduced in 1912 (but Gini did not work with centered distances).[8]

The distance standard deviation is the square root of the distance variance.

The distance correlation [2][3] of two random variables is obtained by dividing their distance covariance by the product of their distance standard deviations. The distance correlation is the square root of

${\displaystyle \operatorname {dCor} ^{2}(X,Y)={\frac {\operatorname {dCov} ^{2}(X,Y)}{\sqrt {\operatorname {dVar} ^{2}(X)\,\operatorname {dVar} ^{2}(Y)}}},}$

and the sample distance correlation is defined by substituting the sample distance covariance and distance variances for the population coefficients above.

For easy computation of sample distance correlation see the dcor function in the energy package for R.[4]

This last property is the most important effect of working with centered distances.

The statistic ${\displaystyle \operatorname {dCov} _{n}^{2}(X,Y)}$ is a biased estimator of ${\displaystyle \operatorname {dCov} ^{2}(X,Y)}$. Under independence of X and Y [9]

${\displaystyle {\begin{aligned}\operatorname {E} [\operatorname {dCov} _{n}^{2}(X,Y)]&={\frac {n-1}{n^{2}}}\left\{(n-2)\operatorname {dCov} ^{2}(X,Y)+\operatorname {E} [\|X-X'\|]\,\operatorname {E} [\|Y-Y'\|]\right\}\\[6pt]&={\frac {n-1}{n^{2}}}\operatorname {E} [\|X-X'\|]\,\operatorname {E} [\|Y-Y'\|].\end{aligned}}}$

An unbiased estimator of ${\displaystyle \operatorname {dCov} ^{2}(X,Y)}$ is given by Székely and Rizzo.[10]

Equality holds in (iv) if and only if one of the random variables X or Y is a constant.