Stein’s method

1.1 Gaussian

The original form, Stein’s lemma (Stein 1972) gives us the Stein operator for the Gaussian distribution in particular. Meckes (2009) explains:

The normal distribution is the unique probability measure \(\mu\) for which \[ \int\left[f^{\prime}(x)-x f(x)\right] \mu(d x)=0 \] for all \(f\) for which the left-hand side exists and is finite. It is useful to think of this in terms of operators, specifically, the operator \(\mathcal{A}_{o}\) defined on \(C^{1}\) functions by \[ \mathcal{A}_{o} f(x)=f^{\prime}(x)-x f(x) \] is called the characterising operator of the standard normal distribution.
This is incredibly useful in probability approximation by Gaussians where it justifies Stein’s method, below. It has apparently been extended to elliptical distributions and exponential families.

Multivariate? Why, yes please. The following lemma of Meckes (2006) gives a second-order characterising operator for the Gaussian distribution on \(\mathbb{R}^{k}\):

For \(f \in C^{1}\left(\mathbb{R}^{k}\right)\), define the gradient of \(f\) by \(\nabla f(x)=\left(\frac{\partial f}{\partial x_{1}}(x), \ldots, \frac{\partial f}{\partial x_{k}}(x)\right)^{t}\). Define the Laplacian of \(f\) by \(\Delta f(x)=\sum_{i=1}^{k} \frac{\partial^{2} f}{\partial x_{i}^{2}}(x)\). Now, let \(Z \sim \mathcal{N}(0_k, \mathrm{I}_k)\).

1. If \(f: \mathbb{R}^{k} \rightarrow \mathbb{R}\) is two times continuously differentiable and compactly supported, then \[ \mathbb{E}[\Delta f(Z)-Z \cdot \nabla f(Z)]=0 \]
2. If \(Y \in \mathbb{R}^{k}\) is a random vector such that \[ \mathbb{E}[\Delta f(Y)-Y \cdot \nabla f(Y)]=0 \] for every \(f \in C^{2}\left(\mathbb{R}^{k}\right)\), then \(\mathcal{L}(Y)=\mathcal{L}(Z) .\)
3. If \(g \in C_{o}^{\infty}\left(\mathbb{R}^{k}\right)\), then the function \[ U_{o} g(x):=\int_{0}^{1} \frac{1}{2 t}[\mathbb{E} g(\sqrt{t} x+\sqrt{1-t} Z)-\mathbb{E} g(Z)] d t \] is a solution to the differential equation \[ \Delta h(x)-x \cdot \nabla h(x)=g(x)-\mathbb{E} g(Z) \]

1.2 Poisson

a.k.a. Stein-Chen (e.g. Chen 1998) \[\mathcal{A}_{o} f(k)=\lambda f(k+1)-k f(k)\]

1.3 Exponential

Ross (2011) mentions

\[ \mathcal{A}_{o} f(x)=f^{\prime}(x)-f(x)+f(0) \]

See also Chatterjee, Fulman, and Röllin (2011) who credits Stein et al. (2004) with > a random variable \(Z\) on \([0, \infty)\) is \(\operatorname{Exp}(1)\) if and only if \(\mathbb{E}\left[f^{\prime}(Z)-f(Z)\right]=-f\left(0^{+}\right)\) for all functions \(f\) in a large class of functions (whose precise definition we do not need).

1.4 Via the method of generators

Barbour (n.d.) gives us a method of creating Stein operators via infinitesimal generators on the way to a Poisson process Stein operator.

Summarised by Gesine Reinert in Barbour and Chen (2005):

The basic idea is to choose the operator \(\mathcal{A}\) to be the generator of a Markov process with stationary distribution \(\mu\). That is, for a homogeneous Markov process \(\left(X_t\right)_{t \geq 0}\), put \(T_t f(x)=\mathbb{E}\left(f\left(X_t\right) \mid X(0)=x\right)\). The generator of the Markov process is defined as \(\mathcal{A} f(x)=\lim _{t \downarrow 0} \frac{1}{t}\left(T_t f(x)-f(x)\right)\). Standard results for generators […]] yield

1. If \(\mu\) is the stationary distribution of the Markov process then \(X \sim \mu\) if and only if \(\mathbb{E} \mathcal{A} f(X)=0\) for all real-valued functions \(f\) for which \(\mathcal{A} f\) is defined.
2. \(T_t h-h=\mathcal{A}\left(\int_0^t T_u h d u\right)\), and, formally taking limits, \[ \int h d \mu-h=\mathcal{A}\left(\int_0^{\infty} T_u h d u\right) \] if the right-hand side exists. Thus the generator approach gives both a Stein equation and a candidate for its solution. One could hence view this approach as a Markov process perturbation technique.

Reassuring example:

The operator \[ \mathcal{A} h(x)=h^{\prime \prime}(x)-x h^{\prime}(x) \] is the generator of the Ornstein-Uhlenbeck process with stationary distribution \(\mathcal{N}(0,1)\). Putting \(f=h^{\prime}\) gives the classical Stein characterization for \(\mathcal{N}(0,1)\)

2.1 Multivariate Gaussian Stein method

Two foundational ones are Chatterjee and Meckes (2008) and Meckes (2009) and there is a kind of introductory user guide in Meckes (2012); The examples are mostly about random projections although the method is much more general. The exchangeable pairs are natural in projections though, you can just switch off your brain and turn the handle to produce results, or easier yet, a computer algebra system that can handle noncommutative algebra can do it for you.

If the papers are too dense, try this friendly lecture, Stein’s Method — The last gadget under the hood.