Transforms of random variates

1 Taylor expansion

Not complicated but subtle (Gustafsson and Hendeby 2012).

Consider a general nonlinear differentiable transformation $g$ and its second-order Taylor expansion. Consider the mapping $g:{\mathbb{R}}^{n_x}\to {\mathbb{R}}^{n_z}$ applied to a variable $x,$ defining $z:=g(x).$ Let $\mathrm{E}(x)={\mu }_x$ and $\mathrm{Var}(x)=P_x.$ The Hessian of the $i^{\text{th }}$ component of $g$ is denoted $g_i^{\prime \prime }.$ $[x_i{]}_i$ is a vector where the $i$th element is $x_i$. We will approximate $z$ using the Taylor expansion, $$z=g\left({\mu }_x\right)+g^{\prime }\left({\mu }_x\right)(x-{\mu }_x)+{\left[\frac{1}{2}{(x-{\mu }_x)}^T{g}_i^{\prime \prime }\left({\mu }_x\right)(x-{\mu }_x)\right]}_i.$$ Leaving aside questions of when this is convergent for now. Then the first moment of $z$ is given by $${\mu }_z=g\left({\mu }_x\right)+\frac{1}{2}{\left[\mathrm{tr}\left(g_i^{\prime \prime }\left({\mu }_x\right)P_x\right)\right]}_i$$ Further, let $x\sim \mathcal{N}({\mu }_x,P_x)$, then the second moment of $z$ is given by $$P_z=g^{\prime }\left({\mu }_x\right)P_x{\left(g^{\prime }\left({\mu }_x\right)\right)}^T+\frac{1}{2}{\left[\mathrm{tr}\left(g_i^{\prime \prime }\left({\mu }_x\right)P_x{g}_j^{\prime \prime }\left({\mu }_x\right)P_x\right)\right]}_{ij}$$ with $i,j=1,\dots ,n_z.$

This is commonly seen in the context of transforms of Gaussians.

2 Unscented transforms

Typically seen for Gaussian RVs. Ebeigbe et al. (2021) claims to have devised a method for more general RVs.

3 Stein’s lemma

As seen in Stein’s method. Gives us the special case of certain exponential RVs (typically Gaussian) under certain matched transforms. Long story.

4 Stochastic Itô-Taylor expansion

Taylor expansions for stochastic processes. See stochastic taylor expansion. tl;dr: Usually more trouble than it is worth.

5 Learnable transforms

See reparameterization trick.

6 References