Exercise 15.10: Bounds on TV Distance
(a)
Applying in order Pinsker-Csiszar-Kullback, Jensen, and the $\log x \leq x - 1$ inequalities \begin{align} \TV (\P , \Q)^2 \leq \frac{1}{2} \int \! p \log \frac{p}{q} \, \sd \nu \leq \frac{1}{2} \log \int \! \frac{p^2}{q} \, \sd \nu \leq \frac{1}{2} \biggl( \int \! \frac{p^2}{q} \, \sd \nu - 1 \biggr) \end{align}
(b)
By (a) \begin{align} \TV (\P_\theta^n , \P_0^n) \leq \frac{1}{2} \biggl[ \biggl(\int \! \frac{p_\theta^2}{p_0} \, \sd \nu \biggr)^n - 1 \biggr] \, . \end{align} It remains to bound the integral. For that, substitute the densities and complete the square to obtain \begin{align} \biggl[ \int \! \frac{1}{\sqrt{2 \pi \sigma^2}} \exp \biggl\lbrace - \frac{1}{2\sigma^2} \bigl[ (x - 2 \theta)^2 - 2 \theta^2 \bigr] \biggr\rbrace \biggr]^n = \exp \biggl\lbrace \biggl( \frac{\sqrt{n} \theta}{\sigma} \biggr)^2 \biggr\rbrace \, . \end{align}
(c)
A tighter bound than required by the exercise can be obtained by convexity of the $\KL$ divergence applied to the Pinsker-Csiszar-Kullback upper bound \begin{align} \KL \bigl( \tfrac{1}{2} \P_{- \theta} + \tfrac{1}{2} \P_{\theta} \, \| \, \P_0 \bigr) \leq \tfrac{1}{2} \KL ( \P_{- \theta} \, \| \, \P_0 ) + \tfrac{1}{2} \KL ( \P_{\theta} \, \| \, \P_0 ) = \frac{\theta^2}{2\sigma^2} \, . \end{align}