Quantum mutual information

For simplicity, it will be assumed that all objects in the article are finite-dimensional.

The definition of quantum mutual entropy is motivated by the classical case. For a probability distribution of two variables p(x, y), the two marginal distributions are

${\displaystyle p(x)=\sum _{y}p(x,y),\qquad p(y)=\sum _{x}p(x,y).}$

The classical mutual information I(X:Y) is defined by

${\displaystyle I(X:Y)=S(p(x))+S(p(y))-S(p(x,y))}$

where S(q) denotes the Shannon entropy of the probability distribution q.

One can calculate directly

${\displaystyle {\begin{aligned}S(p(x))+S(p(y))&=-\left(\sum _{x}p_{x}\log p(x)+\sum _{y}p_{y}\log p(y)\right)\\&=-\left(\sum _{x}\left(\sum _{y'}p(x,y')\log \sum _{y'}p(x,y')\right)+\sum _{y}\left(\sum _{x'}p(x',y)\log \sum _{x'}p(x',y)\right)\right)\\&=-\left(\sum _{x,y}p(x,y)\left(\log \sum _{y'}p(x,y')+\log \sum _{x'}p(x',y)\right)\right)\\&=-\sum _{x,y}p(x,y)\log p(x)p(y)\end{aligned}}}$

So the mutual information is

${\displaystyle I(X:Y)=\sum _{x,y}p(x,y)\log {\frac {p(x,y)}{p(x)p(y)}},}$

Where the logarithm is taken in basis 2 to obtain the mutual information in bits. But this is precisely the relative entropy between p(x, y) and p(x)p(y). In other words, if we assume the two variables x and y to be uncorrelated, mutual information is the discrepancy in uncertainty resulting from this (possibly erroneous) assumption.

It follows from the property of relative entropy that I(X:Y) ≥ 0 and equality holds if and only if p(x, y) = p(x)p(y).

The quantum mechanical counterpart of classical probability distributions are modeled with density matrices.

Consider a quantum system that can be divided into two parts, A and B, such that independent measurements can be made on either part. The state space of the entire quantum system is then the tensor product of the spaces for the two parts.

${\displaystyle H_{AB}:=H_{A}\otimes H_{B}.}$

Let ρ^AB be a density matrix acting on states in H_AB. The von Neumann entropy of a density matrix S(ρ), is the quantum mechanical analogy of the Shannon entropy.

${\displaystyle S(\rho )=-\operatorname {Tr} \rho \log \rho .}$

For a probability distribution p(x,y), the marginal distributions are obtained by integrating away the variables x or y. The corresponding operation for density matrices is the partial trace. So one can assign to ρ a state on the subsystem A by

${\displaystyle \rho ^{A}=\operatorname {Tr} _{B}\;\rho ^{AB}}$

where Tr_B is partial trace with respect to system B. This is the reduced state of ρ^AB on system A. The reduced von Neumann entropy of ρ^AB with respect to system A is

${\displaystyle \;S(\rho ^{A}).}$

S(ρ^B) is defined in the same way.

It can now be seen that the definition of quantum mutual information, corresponding to the classical definition, should be as follows.

${\displaystyle \;I(A\!:\!B):=S(\rho ^{A})+S(\rho ^{B})-S(\rho ^{AB}).}$

Quantum mutual information can be interpreted the same way as in the classical case: it can be shown that

${\displaystyle I(A\!:\!B)=S(\rho ^{AB}\|\rho ^{A}\otimes \rho ^{B})}$

where ${\displaystyle S(\cdot \|\cdot )}$ denotes quantum relative entropy. Note that there is an alternative generalization of mutual information to the quantum case. The difference between the two for a given state is called quantum discord, a measure for the quantum correlations of the state in question.

When the state ${\displaystyle \rho ^{AB}}$ is pure (and thus ${\displaystyle S(\rho ^{AB})=0}$), the mutual information is twice the entanglement entropy of the state:

${\displaystyle I(A\!:\!B)=S(\rho ^{A})+S(\rho ^{B})-S(\rho ^{AB})=S(\rho ^{A})+S(\rho ^{B})=2S(\rho ^{A})}$

A positive quantum mutual information is not necessarily indicative of entanglement, however. A classical mixture of separable states will always have zero entanglement, but can have nonzero QMI, such as

${\displaystyle \rho ^{AB}={\frac {1}{2}}\left(|00\rangle \langle 00|+|11\rangle \langle 11|\right)}$

${\displaystyle I(A\!:\!B)=S(\rho ^{A})+S(\rho ^{B})-S(\rho ^{AB})=S\left({\frac {1}{2}}(|0\rangle \langle 0|+|1\rangle \langle 1|)\right)+S\left({\frac {1}{2}}(|0\rangle \langle 0|+|1\rangle \langle 1|)\right)-S\left({\frac {1}{2}}(|00\rangle \langle 00|+|11\rangle \langle 11|)\right)=\log 2+\log 2-\log 2=\log 2}$

In this case, the state is merely a classically correlated state.

Suppose a system is composed by n subsystems ${\displaystyle A_{1},\dots ,A_{n}}$ then:[1]

${\displaystyle I(A_{1}\!:\!A_{2}:\dots :A_{n})=\sum [S(X_{k_{1}},\,X_{k_{2}},\dots ,X_{k_{n-1}})]-(n-1)S(A_{1},\,A_{2},\,\dots ,\,A_{n})}$

where ${\displaystyle X_{k_{i}}\in \{A_{1},\,A_{2},\,\dots ,\,A_{n}\}}$ and the sum is over all the distinct combinations of the subsystem without repetition.

For example, take ${\displaystyle n=3}$:

${\displaystyle I(A\!:\!B\!:\!C)=S(AB)+S(AC)+S(BC)-2S(ABC)}$

Take now ${\displaystyle n=4}$:

${\displaystyle I(A_{1}\!:\!A_{2}\!:\!A_{3}\!:\!A_{4})=S(A_{1}A_{2}A_{3})+S(A_{1}A_{2}A_{4})+S(A_{1}A_{3}A_{4})+S(A_{2}A_{3}A_{4})-3S(A_{1}A_{2}A_{3}A_{4})}$

Note that what we are actually doing is taking the partial trace over one subsystem per time, take the ${\displaystyle n=4}$ example, in the first term we are tracing over ${\displaystyle A_{4}}$, in the second term the trace is over ${\displaystyle A_{3}}$ and so on.