I have a nonlinear transformation of a random process. What is its distribution?

Related: What is the gradient of the transform? That is the topic of the reparameterization trick.

## Taylor expansion

Not complicated but subtle (Gustafsson and Hendeby 2012).

Consider a general nonlinear differentiable transformation \(g\) and its second order Taylor expansion. Consider the mapping \(g:\mathbb{R}^{n_{x}}\to\mathbb{R}^{n_{z}}\) applied to a variable \(x,\) defining \(z:=g(x).\) Let \(\mathrm{E}(x)=\mu_{x}\) and \(\operatorname{Var}(x)=P_{x}.\) The Hessian of the \(i^{\text {th }}\) component of \(g\) is denoted \(g_{i}^{\prime \prime}.\) \([x_i]_i\) is a vector where the \(i\)th element is \(x_i\). We will approximate \(z\) using the Taylor expansion, \[z=g\left(\mu_{x}\right)+g^{\prime}\left(\mu_{x}\right)\left(x-\mu_{x}\right)+\left[\frac{1}{2}\left(x-\mu_{x}\right)^{T} g_{i}^{\prime \prime}\left(\mu_{x}\right)\left(x-\mu_{x}\right)\right]_{i}.\] Leaving aside questions of when this is convergent for now. Then the first moment of \(z\) is given by \[ \mu_{z}=g\left(\mu_{x}\right)+\frac{1}{2}\left[\operatorname{tr}\left(g_{i}^{\prime \prime}\left(\mu_{x}\right) P_{x}\right)\right]_{i} \] Further, let \(x \sim \mathcal{N}\left(\mu_{x}, P_{x}\right)\), then the second moment of \(z\) is given by \[ P_{z}=g^{\prime}\left(\mu_{x}\right) P_{x}\left(g^{\prime}\left(\mu_{x}\right)\right)^{T}+\frac{1}{2}\left[\operatorname{tr}\left(g_{i}^{\prime \prime}\left(\mu_{x}\right) P_{x} g_{j}^{\prime \prime}\left(\mu_{x}\right) P_{x}\right)\right]_{i j} \] with \(i, j=1, \ldots, n_{z}.\)

## Unscented transform

The great invention of Uhlmann and Julier is the *unscented transform*, which uses a cunningly-chosen non-random empirical sample at so-called *\(\sigma\)-points* to approximate the transformed distribution via its moments.

Mostly seen in the context of Kalman filtering,

What the Unscented Transform does is to replace the mean vector and its associated error covariance matrix with a special set of points with the same mean and covariance. In the case of the mean and covariance representing the current position estimate for a target, the UT is applied to obtain a set of points, referred to as sigma points, to which the full nonlinear equations of motion can be applied directly. In other words, instead of having to derive a linearized approximation, the equations could simply be applied to each of the points as if it were the true state of the target. The result is a transformed set of points, and the mean and covariance of that set represents the estimate of the predicted state of the target.

See, e.g., Roth, Hendeby, and Gustafsson (2016) and a comparison with the Taylor expansion in Gustafsson and Hendeby (2012).

## Stein’s lemma

As seen in Stein’s method. Gives you the special case of certain exponential RVS (typically Gaussian) under certain matched transforms.

## Stochastic Itô-Taylor expansion

Taylor expansions for stochastic processes.
See stochastic taylor expansion.
**tl;dr**: Usually more trouble than it is worth.

## References

*Handbook of Financial Econometrics: Tools and Techniques*, 1–66. Elsevier. https://doi.org/10.1016/B978-0-444-50897-3.50004-3.

*Malaysian Journal of Fundamental and Applied Sciences*13 (3). https://doi.org/10.11113/mjfas.v13n3.633.

*Journal of Economic Dynamics and Control*25 (6-7): 979–99. https://ideas.repec.org/a/eee/dyncon/v25y2001i6-7p979-999.html.

*2008 IEEE International Conference on Acoustics, Speech and Signal Processing*, 3617–20. https://doi.org/10.1109/ICASSP.2008.4518435.

*IEEE Transactions on Signal Processing*60 (2): 545–55. https://doi.org/10.1109/TSP.2011.2172431.

*Lévy Processes: Theory and Applications*, edited by Ole E. Barndorff-Nielsen, Sidney I. Resnick, and Thomas Mikosch, 139–68. Boston, MA: Birkhäuser. https://doi.org/10.1007/978-1-4612-0197-7_7.

*Mathematische Nachrichten*151 (1): 33–50. https://doi.org/10.1002/mana.19911510103.

*Stochastic Analysis and Applications*10 (4): 431–41. https://doi.org/10.1080/07362999208809281.

*Numerical Solution of Stochastic Differential Equations*, edited by Peter E. Kloeden and Eckhard Platen, 161–226. Applications of Mathematics. Berlin, Heidelberg: Springer. https://doi.org/10.1007/978-3-662-12616-5_5.

*Numerical Solution of Stochastic Differential Equations*. Berlin, Heidelberg: Springer Berlin Heidelberg. https://public.ebookcentral.proquest.com/choice/publicfullrecord.aspx?p=3099793.

*Stochastic Analysis and Applications*22 (6): 1553–76. https://doi.org/10.1081/SAP-200029495.

*Journal of Economic Dynamics and Control*28 (4): 755–75. https://doi.org/10.1016/S0165-1889(03)00043-5.

*Introduction to Variance Estimation*. 2nd ed. Statistics for Social and Behavioral Sciences. New York: Springer. https://doi.org/10.1007/978-0-387-35099-8.

## No comments yet. Why not leave one?