# Transforms of random variates

I have a nonlinear transformation of a random process. What is its distribution?

Related: What is the gradient of the transform? That is the topic of the reparameterization trick and other MC grad estimators. This is very commonly seen in the context of transforms of Gaussians when it produces results like the delta method and extended Kalman filtering.

## Taylor expansion

Not complicated but subtle .

Consider a general nonlinear differentiable transformation $$g$$ and its second order Taylor expansion. Consider the mapping $$g:\mathbb{R}^{n_{x}}\to\mathbb{R}^{n_{z}}$$ applied to a variable $$x,$$ defining $$z:=g(x).$$ Let $$\mathrm{E}(x)=\mu_{x}$$ and $$\operatorname{Var}(x)=P_{x}.$$ The Hessian of the $$i^{\text {th }}$$ component of $$g$$ is denoted $$g_{i}^{\prime \prime}.$$ $$[x_i]_i$$ is a vector where the $$i$$th element is $$x_i$$. We will approximate $$z$$ using the Taylor expansion, $z=g\left(\mu_{x}\right)+g^{\prime}\left(\mu_{x}\right)\left(x-\mu_{x}\right)+\left[\frac{1}{2}\left(x-\mu_{x}\right)^{T} g_{i}^{\prime \prime}\left(\mu_{x}\right)\left(x-\mu_{x}\right)\right]_{i}.$ Leaving aside questions of when this is convergent for now. Then the first moment of $$z$$ is given by $\mu_{z}=g\left(\mu_{x}\right)+\frac{1}{2}\left[\operatorname{tr}\left(g_{i}^{\prime \prime}\left(\mu_{x}\right) P_{x}\right)\right]_{i}$ Further, let $$x \sim \mathcal{N}\left(\mu_{x}, P_{x}\right)$$, then the second moment of $$z$$ is given by $P_{z}=g^{\prime}\left(\mu_{x}\right) P_{x}\left(g^{\prime}\left(\mu_{x}\right)\right)^{T}+\frac{1}{2}\left[\operatorname{tr}\left(g_{i}^{\prime \prime}\left(\mu_{x}\right) P_{x} g_{j}^{\prime \prime}\left(\mu_{x}\right) P_{x}\right)\right]_{i j}$ with $$i, j=1, \ldots, n_{z}.$$

This is commonly seen in the context of transforms of Gaussians.

## Unscented transforms

Typically seen for Gaussian RVs. Ebeigbe et al. (2021) claims to have devised a method for more general RVs.

## Stein’s lemma

As seen in Stein’s method. Gives us the special case of certain exponential RVS (typically Gaussian) under certain matched transforms. Long story.

## Stochastic Itô-Taylor expansion

Taylor expansions for stochastic processes. See stochastic taylor expansion. tl;dr: Usually more trouble than it is worth.

## References

Aït-Sahalia, Yacine, Lars Peter Hansen, and José A. Scheinkman. 2010. In Handbook of Financial Econometrics: Tools and Techniques, 1–66. Elsevier.
Ariffin, Noor Amalina Nisa, and Norhayati Rosli. 2017. Malaysian Journal of Fundamental and Applied Sciences 13 (3).
Collard, Fabrice, and Michel Juillard. 2001. Journal of Economic Dynamics and Control 25 (6-7): 979–99.
Easley, Deanna, and Tyrus Berry. 2020. arXiv:2006.13429 [Cs, Math], June.
Ebeigbe, Donald, Tyrus Berry, Michael M. Norton, Andrew J. Whalen, Dan Simon, Timothy Sauer, and Steven J. Schiff. 2021. ArXiv, April, arXiv:2104.01958v1.
Gustafsson, Fredrik, and Gustaf Hendeby. 2008. In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, 3617–20.
———. 2012. IEEE Transactions on Signal Processing 60 (2): 545–55.
Jacob, Niels, and René L. Schilling. 2001. In Lévy Processes: Theory and Applications, edited by Ole E. Barndorff-Nielsen, Sidney I. Resnick, and Thomas Mikosch, 139–68. Boston, MA: Birkhäuser.
Jin, He-hui, Kenneth L Judd, and Hoover Insitution. n.d. “Perturbation Methods for General Dynamic Stochastic Models,” 44.
Kloeden, P. E., and E. Platen. 1991. Mathematische Nachrichten 151 (1): 33–50.
Kloeden, P. E., E. Platen, and I. W. Wright. 1992. Stochastic Analysis and Applications 10 (4): 431–41.
Kloeden, Peter E., and Eckhard Platen. 1992. In Numerical Solution of Stochastic Differential Equations, edited by Peter E. Kloeden and Eckhard Platen, 161–226. Applications of Mathematics. Berlin, Heidelberg: Springer.
———. 2010. Numerical Solution of Stochastic Differential Equations. Berlin, Heidelberg: Springer Berlin Heidelberg.
Lin, Wu, Mohammad Emtiyaz Khan, and Mark Schmidt. 2019. arXiv:1910.13398 [Cs, Stat], October.
Majumdar, Rajeshwari, and Suman Majumdar. 2019. Heliyon 5 (2): e01136.
Papapantoleon, Antonis, and Maria Siopacha. 2010. arXiv:0906.5581 [Math, q-Fin], October.
Rößler, Andreas. 2004. Stochastic Analysis and Applications 22 (6): 1553–76.
Roth, Michael, Gustaf Hendeby, and Fredrik Gustafsson. 2016. “Nonlinear Kalman Filters Explained: A Tutorial on Moment Computations and Sigma Point Methods” 11 (1): 24.
Schmitt-Grohe, Stephanie, and Martın Uribe. n.d. “Perturbation Methods for the Numerical Analysis of DSGE Models: Lecture Notes,” 38.
Schmitt-Grohé, Stephanie, and Martı́n Uribe. 2004. Journal of Economic Dynamics and Control 28 (4): 755–75.
Schoutens, Wim, K U Leuven, and Michael Studer. 2001. February, 24.
Wolter, Kirk M. 2007. Introduction to Variance Estimation. 2nd ed. Statistics for Social and Behavioral Sciences. New York: Springer.

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.