Generalized relative entropy

To motivate the definition of the ${\displaystyle \epsilon }$ -relative entropy ${\displaystyle D^{\epsilon }(\rho ||\sigma )}$ , consider the information processing task of hypothesis testing. In hypothesis testing, we wish to devise a strategy to distinguish between two density operators ${\displaystyle \rho }$  and ${\displaystyle \sigma }$ . A strategy is a POVM with elements ${\displaystyle Q}$  and ${\displaystyle I-Q}$ . The probability that the strategy produces a correct guess on input ${\displaystyle \rho }$  is given by ${\displaystyle \operatorname {Tr} (\rho Q)}$  and the probability that it produces a wrong guess is given by ${\displaystyle \operatorname {Tr} (\sigma Q)}$ . ${\displaystyle \epsilon }$ -relative entropy captures the minimum probability of error when the state is ${\displaystyle \sigma }$ , given that the success probability for ${\displaystyle \rho }$  is at least ${\displaystyle \epsilon }$ .

For ${\displaystyle \epsilon \in (0,1)}$ , the ${\displaystyle \epsilon }$ -relative entropy between two quantum states${\displaystyle \rho }$  and ${\displaystyle \sigma }$  is defined as

${\displaystyle D^{\epsilon }(\rho ||\sigma )=-\log {\frac {1}{\epsilon }}\min\{\langle Q,\sigma \rangle |0\leq Q\leq I{\text{ and }}\langle Q,\rho \rangle \geq \epsilon \}~.}$ 

From the definition, it is clear that ${\displaystyle D^{\epsilon }(\rho ||\sigma )\geq 0}$ . This inequality is saturated if and only if ${\displaystyle \rho =\sigma }$ , as shown below.

Suppose the trace distance between two density operators ${\displaystyle \rho }$  and ${\displaystyle \sigma }$  is

${\displaystyle ||\rho -\sigma ||_{1}=\delta ~.}$ 

For ${\displaystyle 0<\epsilon <1}$ , it holds that

a) ${\displaystyle \log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}\quad \leq \quad D^{\epsilon }(\rho ||\sigma )\quad \leq \quad \log {\frac {\epsilon }{\epsilon -\delta }}~.}$ 

In particular, this implies the following analogue of the Pinsker inequality^[1]

b) ${\displaystyle {\frac {1-\epsilon }{\epsilon }}||\rho -\sigma ||_{1}\quad \leq \quad D^{\epsilon }(\rho ||\sigma )~.}$ 

Furthermore, the proposition implies that for any ${\displaystyle \epsilon \in (0,1)}$ , ${\displaystyle D^{\epsilon }(\rho ||\sigma )=0}$  if and only if ${\displaystyle \rho =\sigma }$ , inheriting this property from the trace distance. This result and its proof can be found in Dupuis et al.^[2]

Upper bound: Trace distance can be written as

${\displaystyle ||\rho -\sigma ||_{1}=\max _{0\leq Q\leq 1}\operatorname {Tr} (Q(\rho -\sigma ))~.}$ 

This maximum is achieved when ${\displaystyle Q}$  is the orthogonal projector onto the positive eigenspace of ${\displaystyle \rho -\sigma }$ . For any POVM element ${\displaystyle Q}$  we have

${\displaystyle \operatorname {Tr} (Q(\rho -\sigma ))\leq \delta }$ 

so that if ${\displaystyle \operatorname {Tr} (Q\rho )\geq \epsilon }$ , we have

${\displaystyle \operatorname {Tr} (Q\sigma )~\geq ~\operatorname {Tr} (Q\rho )-\delta ~\geq ~\epsilon -\delta ~.}$ 

From the definition of the ${\displaystyle \epsilon }$ -relative entropy, we get

${\displaystyle 2^{-D^{\epsilon }(\rho ||\sigma )}\geq {\frac {\epsilon -\delta }{\epsilon }}~.}$ 

Lower bound: Let ${\displaystyle Q}$  be the orthogonal projection onto the positive eigenspace of ${\displaystyle \rho -\sigma }$ , and let ${\displaystyle {\bar {Q}}}$  be the following convex combination of ${\displaystyle I}$  and ${\displaystyle Q}$ :

${\displaystyle {\bar {Q}}=(\epsilon -\mu )I+(1-\epsilon +\mu )Q}$ 

where ${\displaystyle \mu ={\frac {(1-\epsilon )\operatorname {Tr} (Q\rho )}{1-\operatorname {Tr} (Q\rho )}}~.}$ 

This means

${\displaystyle \mu =(1-\epsilon +\mu )\operatorname {Tr} (Q\rho )}$ 

and thus

${\displaystyle \operatorname {Tr} ({\bar {Q}}\rho )~=~(\epsilon -\mu )+(1-\epsilon +\mu )\operatorname {Tr} (Q\rho )~=~\epsilon ~.}$ 

Moreover,

${\displaystyle \operatorname {Tr} ({\bar {Q}}\sigma )~=~\epsilon -\mu +(1-\epsilon +\mu )\operatorname {Tr} (Q\sigma )~.}$ 

Using ${\displaystyle \mu =(1-\epsilon +\mu )\operatorname {Tr} (Q\rho )}$ , our choice of ${\displaystyle Q}$ , and finally the definition of ${\displaystyle \mu }$ , we can re-write this as

${\displaystyle \operatorname {Tr} ({\bar {Q}}\sigma )~=~\epsilon -(1-\epsilon +\mu )\operatorname {Tr} (Q\rho )+(1-\epsilon +\mu )\operatorname {Tr} (Q\sigma )}$ 

${\displaystyle ~=~\epsilon -{\frac {(1-\epsilon )\delta }{1-\operatorname {Tr} (Q\rho )}}~\leq ~\epsilon -(1-\epsilon )\delta ~.}$ 

Hence

${\displaystyle D^{\epsilon }(\rho ||\sigma )\geq \log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}~.}$ 

To derive this Pinsker-like inequality, observe that

${\displaystyle \log {\frac {\epsilon }{\epsilon -(1-\epsilon )\delta }}~=~-\log \left(1-{\frac {(1-\epsilon )\delta }{\epsilon }}\right)~\geq ~\delta {\frac {1-\epsilon }{\epsilon }}~.}$ 

A fundamental property of von Neumann entropy is strong subadditivity. Let ${\displaystyle S(\sigma )}$  denote the von Neumann entropy of the quantum state ${\displaystyle \sigma }$ , and let ${\displaystyle \rho _{ABC}}$  be a quantum state on the tensor product Hilbert space ${\displaystyle {\mathcal {H}}_{A}\otimes {\mathcal {H}}_{B}\otimes {\mathcal {H}}_{C}}$ . Strong subadditivity states that

${\displaystyle S(\rho _{ABC})+S(\rho _{B})\leq S(\rho _{AB})+S(\rho _{BC})}$ 

where ${\displaystyle \rho _{AB},\rho _{BC},\rho _{B}}$  refer to the reduced density matrices on the spaces indicated by the subscripts. When re-written in terms of mutual information, this inequality has an intuitive interpretation; it states that the information content in a system cannot increase by the action of a local quantum operation on that system. In this form, it is better known as the data processing inequality, and is equivalent to the monotonicity of relative entropy under quantum operations:^[3]

${\displaystyle S(\rho ||\sigma )-S({\mathcal {E}}(\rho )||{\mathcal {E}}(\sigma ))\geq 0}$ 

for every CPTP map ${\displaystyle {\mathcal {E}}}$ , where ${\displaystyle S(\omega ||\tau )}$  denotes the relative entropy of the quantum states ${\displaystyle \omega ,\tau }$ .

It is readily seen that ${\displaystyle \epsilon }$ -relative entropy also obeys monotonicity under quantum operations:^[4]

${\displaystyle D^{\epsilon }(\rho ||\sigma )\geq D^{\epsilon }({\mathcal {E}}(\rho )||{\mathcal {E}}(\sigma ))}$ ,

for any CPTP map ${\displaystyle {\mathcal {E}}}$ . To see this, suppose we have a POVM ${\displaystyle (R,I-R)}$  to distinguish between ${\displaystyle {\mathcal {E}}(\rho )}$  and ${\displaystyle {\mathcal {E}}(\sigma )}$  such that ${\displaystyle \langle R,{\mathcal {E}}(\rho )\rangle =\langle {\mathcal {E}}^{\dagger }(R),\rho \rangle \geq \epsilon }$ . We construct a new POVM ${\displaystyle ({\mathcal {E}}^{\dagger }(R),I-{\mathcal {E}}^{\dagger }(R))}$  to distinguish between ${\displaystyle \rho }$  and ${\displaystyle \sigma }$ . Since the adjoint of any CPTP map is also positive and unital, this is a valid POVM. Note that ${\displaystyle \langle R,{\mathcal {E}}(\sigma )\rangle =\langle {\mathcal {E}}^{\dagger }(R),\sigma \rangle \geq \langle Q,\sigma \rangle }$ , where ${\displaystyle (Q,I-Q)}$  is the POVM that achieves ${\displaystyle D^{\epsilon }(\rho ||\sigma )}$ . Not only is this interesting in itself, but it also gives us the following alternative method to prove the data processing inequality.^[2]

By the quantum analogue of the Stein lemma,^[5]

${\displaystyle \lim _{n\rightarrow \infty }{\frac {1}{n}}D^{\epsilon }(\rho ^{\otimes n}||\sigma ^{\otimes n})=\lim _{n\rightarrow \infty }{\frac {-1}{n}}\log \min {\frac {1}{\epsilon }}\operatorname {Tr} (\sigma ^{\otimes n}Q)}$ 

${\displaystyle =D(\rho ||\sigma )-\lim _{n\rightarrow \infty }{\frac {1}{n}}\left(\log {\frac {1}{\epsilon }}\right)}$ 

${\displaystyle =D(\rho ||\sigma )~,}$ 

where the minimum is taken over ${\displaystyle 0\leq Q\leq 1}$  such that ${\displaystyle \operatorname {Tr} (Q\rho ^{\otimes n})\geq \epsilon ~.}$ 

Applying the data processing inequality to the states ${\displaystyle \rho ^{\otimes n}}$  and ${\displaystyle \sigma ^{\otimes n}}$  with the CPTP map ${\displaystyle {\mathcal {E}}^{\otimes n}}$ , we get

${\displaystyle D^{\epsilon }(\rho ^{\otimes n}||\sigma ^{\otimes n})~\geq ~D^{\epsilon }({\mathcal {E}}(\rho )^{\otimes n}||{\mathcal {E}}(\sigma )^{\otimes n})~.}$ 

Dividing by ${\displaystyle n}$  on either side and taking the limit as ${\displaystyle n\rightarrow \infty }$ , we get the desired result.

• Entropic value at risk
• Quantum relative entropy
• Strong subadditivity
• Classical information theory
• Min-entropy