Exercise 13.7: Rates for Polynomial Regression
This is just linear regression. Hence, $\delta_n^2 \propto \sigma^2 m/n$, so apply Theorem 13.5 with $t = \log(n) \delta_n \ge \delta_n$ to get the result.
alternative solution
Since \(f_\theta - f_{\theta^\star} = \langle \theta - \theta^\star, \phi(\cdot) \rangle\) where \(\phi(x) = [1, x, \ldots, x^{m - 1}]\), the function class is star-shaped. By Theorem 13.5, for any valid \(\delta_n\)and \(t \geq \delta_n\), we therefore have \begin{equation} \P(\| \hat{f} - f \|_n^2 \geq t \delta_n) \leq \exp \bigl\lbrace - \tfrac{n t \delta_n}{2\sigma^2} \bigr\rbrace \, . \end{equation}
Let \(\Phi = [\phi(x_1) ; \ldots ; \phi(x_n)] \in \R^{n \times m}\), so that \begin{equation} \| f_{\hat{\theta}} - f_{\theta^\star} \|_n^2 = \frac{\| \Phi (\hat{\theta} - \theta^\star) \|_2^2}{n} \, . \end{equation} Since the VC dimension of order \(m - 1\) polynomials is \(m\), and \(n^{-1} \langle w , g(x_{1:n}) \rangle\) is \(\| g \|_n\)-subgaussian, we can apply the Sauer-Shelah result to obtain \begin{equation} \card(\F^\star(x_{1:n})) \leq \biggl( \frac{e n}{m} \biggr)^m \, . \end{equation} This implies the standard bound \begin{equation} \E \biggl[ \sup_{\| g \|_n \leq \delta} \biggl| \frac{1}{n} \sum_{i = 1}^n w_i g(x_i) \biggr| \biggr] \leq \delta \sqrt{ \tfrac{m}{n} \log \tfrac{e n} {m} } \, . \end{equation} Take \(t = \delta_n = \Theta(\sigma \sqrt{\tfrac{m}{n} \log \tfrac{e n} {m}})\) to conclude.