Jensen Gap

Related to generic concentration of measure, but about the expectation of functions.

This note exists simply because I had not heard about this concept before, but it ended up being really useful even to name it.

Remember the classic Jensen inequality, where for some convex function $f$ and random variable $X$ we have $$f(\mathbb{E}[X])\le \mathrm{E}[f(X)].$$

The Jensen Gap is the value $$J(f,X):=f(\mathbb{E}[X])-\mathrm{E}[f(X)].$$ for a given $X$ and (not necessarily convex) $f$.

Amazingly, we can sometimes say things about how big this gap is. For continuous $f$ and $X$ with $f$ differentiable, the most impressive results are (Abramovich and Persson 2016; Gao, Sitharam, and Roitberg 2020).

If $X$ is discrete and $f$ may or may not be differentiable, we can still say some things — see Simic (2008).