Generalized relative entropy

To motivate the definition of the ${\displaystyle \epsilon }$-relative entropy ${\displaystyle D^{\epsilon }(\rho ||\sigma )}$, consider the information processing task of hypothesis testing. In hypothesis testing, we wish to devise a strategy to distinguish between two density operators ${\displaystyle \rho }$ and ${\displaystyle \sigma }$. A strategy is a POVM with elements ${\displaystyle Q}$ and ${\displaystyle I-Q}$. The probability that the strategy produces a correct guess on input ${\displaystyle \rho }$ is given by ${\displaystyle \operatorname {Tr} (\rho Q)}$ and the probability that it produces a wrong guess is given by ${\displaystyle \operatorname {Tr} (\sigma Q)}$. ${\displaystyle \epsilon }$-relative entropy captures the minimum probability of error when the state is ${\displaystyle \sigma }$, given that the success probability for ${\displaystyle \rho }$ is at least ${\displaystyle \epsilon }$.

For ${\displaystyle \epsilon \in (0,1)}$, the ${\displaystyle \epsilon }$-relative entropy between two quantum states${\displaystyle \rho }$ and ${\displaystyle \sigma }$ is defined as

${\displaystyle D^{\epsilon }(\rho ||\sigma )=-\log {\frac {1}{\epsilon }}\min\{\langle Q,\sigma \rangle |0\leq Q\leq I{\text{ and }}\langle Q,\rho \rangle \geq \epsilon \}~.}$

From the definition, it is clear that ${\displaystyle D^{\epsilon }(\rho ||\sigma )\geq 0}$. This inequality is saturated if and only if ${\displaystyle \rho =\sigma }$, as shown below.

Suppose the trace distance between two density operators ${\displaystyle \rho }$ and ${\displaystyle \sigma }$ is

${\displaystyle ||\rho -\sigma ||_{1}=\delta ~.}$

For ${\displaystyle 0<\epsilon <1}$, it holds that

a) ${\displaystyle \log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}\quad \leq \quad D^{\epsilon }(\rho ||\sigma )\quad \leq \quad \log {\frac {\epsilon }{\epsilon -\delta }}~.}$

In particular, this implies the following analogue of the Pinsker inequality[1]

b) ${\displaystyle {\frac {1-\epsilon }{\epsilon }}||\rho -\sigma ||_{1}\quad \leq \quad D^{\epsilon }(\rho ||\sigma )~.}$

Furthermore, the proposition implies that for any ${\displaystyle \epsilon \in (0,1)}$, ${\displaystyle D^{\epsilon }(\rho ||\sigma )=0}$ if and only if ${\displaystyle \rho =\sigma }$, inheriting this property from the trace distance. This result and its proof can be found in Dupuis et al.[2]

A fundamental property of von Neumann entropy is strong subadditivity. Let ${\displaystyle S(\sigma )}$ denote the von Neumann entropy of the quantum state ${\displaystyle \sigma }$, and let ${\displaystyle \rho _{ABC}}$ be a quantum state on the tensor product Hilbert space ${\displaystyle {\mathcal {H}}_{A}\otimes {\mathcal {H}}_{B}\otimes {\mathcal {H}}_{C}}$. Strong subadditivity states that

${\displaystyle S(\rho _{ABC})+S(\rho _{B})\leq S(\rho _{AB})+S(\rho _{BC})}$

where ${\displaystyle \rho _{AB},\rho _{BC},\rho _{B}}$ refer to the reduced density matrices on the spaces indicated by the subscripts. When re-written in terms of mutual information, this inequality has an intuitive interpretation; it states that the information content in a system cannot increase by the action of a local quantum operation on that system. In this form, it is better known as the data processing inequality, and is equivalent to the monotonicity of relative entropy under quantum operations:[3]

${\displaystyle S(\rho ||\sigma )-S({\mathcal {E}}(\rho )||{\mathcal {E}}(\sigma ))\geq 0}$

for every CPTP map ${\displaystyle {\mathcal {E}}}$, where ${\displaystyle S(\omega ||\tau )}$ denotes the relative entropy of the quantum states ${\displaystyle \omega ,\tau }$.

It is readily seen that ${\displaystyle \epsilon }$-relative entropy also obeys monotonicity under quantum operations:[4]

${\displaystyle D^{\epsilon }(\rho ||\sigma )\geq D^{\epsilon }({\mathcal {E}}(\rho )||{\mathcal {E}}(\sigma ))}$,

for any CPTP map ${\displaystyle {\mathcal {E}}}$. To see this, suppose we have a POVM ${\displaystyle (R,I-R)}$ to distinguish between ${\displaystyle {\mathcal {E}}(\rho )}$ and ${\displaystyle {\mathcal {E}}(\sigma )}$ such that ${\displaystyle \langle R,{\mathcal {E}}(\rho )\rangle =\langle {\mathcal {E}}^{\dagger }(R),\rho \rangle \geq \epsilon }$. We construct a new POVM ${\displaystyle ({\mathcal {E}}^{\dagger }(R),I-{\mathcal {E}}^{\dagger }(R))}$ to distinguish between ${\displaystyle \rho }$ and ${\displaystyle \sigma }$. Since the adjoint of any CPTP map is also positive and unital, this is a valid POVM. Note that ${\displaystyle \langle R,{\mathcal {E}}(\sigma )\rangle =\langle {\mathcal {E}}^{\dagger }(R),\sigma \rangle \geq \langle Q,\sigma \rangle }$, where ${\displaystyle (Q,I-Q)}$ is the POVM that achieves ${\displaystyle D^{\epsilon }(\rho ||\sigma )}$. Not only is this interesting in itself, but it also gives us the following alternative method to prove the data processing inequality.[2]

By the quantum analogue of the Stein lemma,[5]

${\displaystyle \lim _{n\rightarrow \infty }{\frac {1}{n}}D^{\epsilon }(\rho ^{\otimes n}||\sigma ^{\otimes n})=\lim _{n\rightarrow \infty }{\frac {-1}{n}}\log \min {\frac {1}{\epsilon }}\operatorname {Tr} (\sigma ^{\otimes n}Q)}$

${\displaystyle =D(\rho ||\sigma )-\lim _{n\rightarrow \infty }{\frac {1}{n}}\left(\log {\frac {1}{\epsilon }}\right)}$

${\displaystyle =D(\rho ||\sigma )~,}$

where the minimum is taken over ${\displaystyle 0\leq Q\leq 1}$ such that ${\displaystyle \operatorname {Tr} (Q\rho ^{\otimes n})\geq \epsilon ~.}$

Applying the data processing inequality to the states ${\displaystyle \rho ^{\otimes n}}$ and ${\displaystyle \sigma ^{\otimes n}}$ with the CPTP map ${\displaystyle {\mathcal {E}}^{\otimes n}}$, we get

${\displaystyle D^{\epsilon }(\rho ^{\otimes n}||\sigma ^{\otimes n})~\geq ~D^{\epsilon }({\mathcal {E}}(\rho )^{\otimes n}||{\mathcal {E}}(\sigma )^{\otimes n})~.}$

Dividing by ${\displaystyle n}$ on either side and taking the limit as ${\displaystyle n\rightarrow \infty }$, we get the desired result.

• Entropic value at risk
• Quantum relative entropy
• Strong subadditivity
• Classical information theory
• Min-entropy