Exercise 15.6: Pinsker–Csiszar–Kullback Inequality
(a)
The claim is trivially true if $p \in \lbrace 0, 1 \rbrace$. Otherwise fix a $p \in (0, 1)$ and define \begin{align} f(q) = \KL (p \, \| \, q) - 2 (p - q)^2 \, . \end{align} Then \begin{align} f’(q) &= - \tfrac{p}{q} + \tfrac{1-p}{1-q} + 4 (p - q) \newline &= \tfrac{q \pm pq - p}{q(1-q)} - 4 (q-p) = (q - p) \bigl[ \tfrac{1}{q (1 - q)} - 4 \bigr] \, , \end{align} where the latter factor on the r.h.s. is non-negative (equality to zero at $q = 1/2$). Hence $f(p) = 0$, and $f’(q) \leq 0$ when $q \leq p$, and $f’(q) \geq 0$ otherwise, which implies $f$ attains its minimum at $p$, which in turn yields the desired inequality.
(b)
Using the fact $\KL ( \mathbb{P} \, \| \, \mathbb{Q}) \geq \KL (g_\# \mathbb{P} \, \| \, g_\# \mathbb{Q})$ for any measurable function $g$, we take $g = \ind_A$ for $A = \lbrace p \geq q \rbrace$ to obtain \begin{align} \KL (\mathbb{P} \, \| \, \mathbb{Q}) \geq \mathbb{P} (A) \log \tfrac{\mathbb{P}(A)}{\mathbb{Q}(A)} + (1 - \mathbb{P} (A)) \log \tfrac{1 - \mathbb{P}(A)}{1 - \mathbb{Q}(A)} \, . \end{align} The result then follows by combining (a) with $\| \mathbb{P} - \mathbb{Q} \|_{\TV} = \sup_{A} |\mathbb{P}(A) - \mathbb{Q}(A)|$.