Exercise 5.12: Gaussian Contraction Inequality

chapter 5

Let $X_{\varphi (\theta )}=⟨\varphi (\theta ),w⟩$, where $w\sim \mathcal{N}(0,I_d)$ is a shared standard normal random vector, so that $\mathcal{G}(\varphi (\mathbb{T}))=\mathbb{E}[\underset{\theta }{sup}{X}_{\varphi (\theta )}]$. By linearity of expectation $\mathbb{E}[X_{\varphi (\theta )}]=0$, and the associated squared metric $\mathbb{E}[(X_{\varphi (\theta )}-X_{\varphi ({\theta }^{\prime })}{)}^2]=\Vert \varphi (\theta )-\varphi ({\theta }^{\prime }){\Vert }_2^2$ is upper bounded by $\Vert \theta -{\theta }^{\prime }{\Vert }_2^2$ due to the 1-Lipschitz assumption. Hence for any finite $G\subseteq \mathbb{T}$ $$\begin{array}{}\text{(1)}& \mathbb{E}[\underset{\theta \in G}{sup}⟨\varphi (\theta ),w⟩]\le \mathbb{E}[\underset{\theta \in G}{sup}⟨\theta ,w⟩],\end{array}$$ by the Sudakov-Fernique inequality (Theorem 5.27). Applying the monotone convergence theorem, first on the r.h.s., then on the l.h.s., yields the result.