Jensen Gap

Related to generic concentration of measure, but about the expectation of functions.

This note exists simply because I had not heard about this concept before, but it ended up being really useful even to name it.

Remember the classic Jensen inequality, where for some convex function \(f\) and random variable \(X\) we have \[ f \left(\mathbb{E} [X]\right)\leq \operatorname {E} \left[f (X)\right]. \]

The Jensen Gap is the value \[ J(f,X) := f \left(\mathbb{E} [X]\right)- \operatorname {E} \left[f (X)\right]. \] for a given \(X\) and (not necessarily convex) \(f\).

Amazingly, we can sometimes say things about how big this gap is. For continuous \(f\) and \(X\) with \(f\) differentiable, the most impressive results are (Abramovich and Persson 2016; Gao, Sitharam, and Roitberg 2020).

If \(X\) is discrete and \(f\) may or may not be differentiable, we can still say some things — see Simic (2008).