Exercise 2.5: Sub-gaussian Bounds and Means and Variances
(a)
Recall that the derivative of the moment-generating function at \(\lambda = 0\) is equal to \(\E[X]\). For \(\e > 0\), \begin{equation} \frac{1}{\e}(\E[e^{\e X}] - 1) \le \frac{1}{\e}\parens{1 + \frac12\e^2 \sigma^2 + \e \mu + o(\e) - 1} = \frac{1}{2} \e \sigma^2 + \mu + o(1), \end{equation} so take \(\e \downarrow 0\) to find \(\E[X] \le \mu\). The other inequality is similarly found by considering \(1 - \E[e^{-\e X}]\). Thus \(\E[X] = \mu\).
(b)
Again, perform a series expansion: \begin{equation} \E[e^{\lambda (X - \mu)}] = 1 + \frac12 \lambda^2 \E[(X - \mu)^2] + o(\lambda^2) \le 1 + \frac12 \lambda^2 \sigma^2 + o(\lambda^2), \end{equation} so subtract \(1\), divide by \(\lambda^2 > 0\), and take \(\lambda \to 0\) to find \(\V[X] \le \sigma^2\).
(c)
Consider \(X \sim \ber(p)\). Then \(\V[X] = p(1-p)\). It holds that \(\sigma^2\) needs to satisfy \begin{equation} (1 - p) + p e^{\lambda} \le e^{\frac12 \sigma^2 \lambda^2 + p \lambda} \implies \frac{2}{\lambda^2} \parens{ \log\parens{(1 - p) + p e^{\lambda}} - p \lambda } \le \sigma^2. \end{equation} For example, if \(p = \frac14\), then at \(\lambda = 1\) we find \begin{equation} 0.2 < \text{l.h.s.} \le \sigma^2, \end{equation} but \(\V[X] = 0.1875 < 0.2.\) This is a counterexample.