Exercise 13.1: Characterisation of the Bayes Least-Squares Estimate

chapter 13

Using the tower property, note that \begin{equation} \E[(Y - f(X))^2] = \E[(Y - \E[Y \cond X])^2] + \E[(\E[Y\cond X] - f(X))^2], \end{equation} so $f^*(x) = \E[Y \cond X = x]$ and \begin{equation} \E[(Y - f(X))^2] - \E[(Y - f^*(X))^2] = \norm{f - f^*}_2^2. \end{equation}