Exercise 3.6: Entropy Rescaling
Both results follow by the MGF identities \begin{align} \varphi_{aX + c} (\lambda) = \E [e^{\lambda (aX + c)}] = e^{\lambda c} \varphi_{X} (a \lambda) \, , \end{align} and \begin{align} \varphi_{aX + c}’(\lambda) = \E [(aX + c)e^{\lambda (aX + c)}] = e^{\lambda c} \left[ a \varphi_{X}’(a \lambda) + c \varphi_{X}(a \lambda) \right] \, . \end{align}
(a)
(i)
Bound for \(\tilde{X}\) implies bound for \(X\):
\begin{align} \mathbb{H}(e^{\lambda X}) &\leq \lambda^2 e^{\lambda \E [X]} \left[ b \varphi_{\tilde{X}}’ (\lambda) + \varphi_{\tilde{X}}(\lambda) \sigma^2 \right] \newline &= \lambda^2 \left\lbrace b [ \varphi_{X}’(\lambda) - \varphi_X(\lambda) \E [X] ] + \varphi_{X}(\lambda) \sigma^2 \right\rbrace \, . \end{align}
(ii)
Bound for \(X\) implies bound for \(\tilde{X}\):
\begin{align} \mathbb{H}(e^{\lambda \tilde{X}}) &\leq \lambda^2 e^{-\lambda \E [X]} \left\lbrace b \varphi_{X}’ (\lambda) + \varphi_{X}(\lambda) [ \sigma^2 - b \E [X] ] \right\rbrace \newline &= \lambda^2 \left\lbrace b [ \varphi_{\tilde{X}}’(\lambda) + \varphi_{\tilde{X}}(\lambda) \E [X] ] + \varphi_{\tilde{X}}(\lambda) [ \sigma^2 - b \E [X] ] \right\rbrace \, . \end{align}
(b)
(i)
Bound for \(\tilde{X}\) implies bound for \(X\):
\begin{align} \mathbb{H}\bigl(e^{\lambda X / b}\bigr) \overset{\lambda \in [0, 1]}{\leq} \tfrac{\lambda^2}{b^2} \left\lbrace b \E \left[ b \tfrac{X}{b} e^{\lambda X / b} \right] + \sigma^2 \E \bigl[ e^{\lambda X / b} \bigr] \right\rbrace \, . \end{align}
(ii)
Bound for \(X\) implies bound for \(\tilde{X}\):
\begin{align} \mathbb{H}\bigl(e^{\lambda b \tilde{X}}\bigr) \overset{\lambda \in [0, 1 / b]}{\leq} \lambda^2 b^2 \left\lbrace b \E \left[ \tfrac{X}{b} e^{\lambda b X / b} \right] + \tfrac{\sigma^2}{b^2} \E \bigl[ e^{\lambda b X / b} \bigr] \right\rbrace \, . \end{align}