HDS

Exercise 2.4: Sharp Sub-Gaussian Parameter for Bounded Random Variable

chapter 2

Consider a random variable X with mean μ=E[X], and such that, for some scalars b>a,X[a,b] almost surely.

(a)

To begin with, ψ(0)=log1=0. For ψ(0), we know that the derivative of the MGF equals μ, so

(1)ψ(0)=μE[e0X]=μ.

(b)

The identity for ψ(λ) follows from the chain rule. For the upper bound, observe that we can define a new distribution Qλ by taking eλX/E[eλX] to be its Radon–Nikodym derivative (density) with respect to the distribution of X. Hence establishing a bound on ψ(λ) is equivalent to bounding the supremum over variances of random variables XλQλ.

Taking m:=12(a+b), using that the mean minimises the mean squared error, and using that Xλ[a,b] a.s. for all λ,

(2)supλV(Xλ)=supλE[(XλE[Xλ])2]supλE[(Xλm)2](bm)2=(ba)24.

(c)

Taking a Taylor expansion of ψ(λ) at λ=0,

(3)ψ(λ)=ψ(0)+λψ(0)+λ22ψ(ε),

for some ε(0,λ). Substituting the results from (a) and (b), ψ(λ)λμ+λ22(ba)24 as desired.

Published on 25 July 2020.