Exercise 5.10: Concentration of Gaussian Suprema
To avoid issues of measurability, we interpret \begin{align} \E [ \phi ( \sup_{\theta \in \mathbb{T}} X_\theta - \E [\sup_{\theta \in \mathbb{T}} X_\theta] ) ] = \sup \bigl\lbrace \E [ \phi( \sup_{\theta \in G} X_\theta - \E [\sup_{\theta \in G} X_\theta] ) ] \colon G \subseteq \mathbb{T} \, , |G| < \infty \bigr\rbrace \, , \end{align} for any Borel measurable \(\phi \colon \R \to \R\), analogously to Section 4.4.
For a fixed finite subset \(G \subseteq \mathbb{T}\), \(X_G = [X_\theta]_{\theta \in G}\) is an \(\R^{|G|}\) valued \(\gauss(0, K_G)\) distributed random variable by the Gaussian process definition. We can therefore take the eigendecomposition \(K_G = Q \Lambda Q^\top\), and observe \(X_G\) equals \(Q \Lambda^{1/2} Z\), \(Z \sim \gauss(0, I_{|G|})\), in distribution. Noting that \begin{align} z \mapsto \max_{i \in [|G|]} [Q \Lambda^{1/2} z]_i \end{align} is \(\sqrt{\lambda_\text{max}(G)}\)-Lipschitz, \(\lambda_\text{max}(G) = \max_i \Lambda_{ii}\), Theorem 2.26 yields \begin{align} \P \bigl( \bigl| \sup_{G} X_\theta - \E [\sup_{G} X_\theta] \bigr| \geq \delta \bigr) \leq 2 e^{- \frac{\delta^2}{2 \lambda_\text{max}(G)}} \, . \end{align}
Observing \(\sup_{G \subseteq \mathbb{T}} \lambda_\text{max}(G) = \sup_{\theta \in \mathbb{T}} \V (X_\theta) = \sigma^2\), and taking \begin{align} \phi \colon x \mapsto \ind \lbrace | x | \geq \delta \rbrace \, , \end{align} in the aforementioned definition, yields the result.