# Chaos expansions

Polynomial chaos, generalized polynomial chaos, arbitrary chaos etc

May 21, 2020 — February 15, 2021

Bayes
Hilbert space
Lévy processes
Markov processes
probability
SDEs
stochastic processes

Placeholder, for a topic which has a slightly confusing name. To explore: Connection to/difference from other methods of keeping track of evolution of uncertainty in dynamical systems. C&C Gaussian process regression as used in Gratiet, Marelli, and Sudret (2016), functional data analysis etc.

This is not the same thing as chaos in the sense of the deterministic chaos made famous by dynamical systems theory and fractal t-shirts. Different term

## 1 Polynomial chaos expansion

Ground Zero, the original famous one. Wikipedia has an inscrutable introduction which I can make no sense of.

Popular introductions to the concept seem to be . For my money, the most direct is O’Hagan (2013) which runs thusly: We define an inner product $\left\langle \psi, \phi\right\rangle=\int \psi(\xi) \phi(\xi) p_{\xi}(\xi) d \xi$ with respect to the probability density function $$p_{\xi}$$ of some random variate $$\Xi$$ which we call the germ.

Suppose we have chosen a functional basis comprising polynomials $$\psi_{0}=1, \psi_{1}, \psi_{2}, \ldots,$$ where $$\psi_{j}$$ is a polynomial of order $$j$$ and where they satisfy the orthogonality condition that for all $$j \neq k$$ $\left\langle\psi_{j}, \psi_{k}\right\rangle=0.$

We write $X=f(\Xi)=\sum_{j=0}^{\infty} x_{j} \psi_{j}(\Xi)$ The combination of $$x_{j}$$ (the mode strength) and $$\psi_{j}$$ (the mode function) is called the $$j$$ -th mode. By orthogonality, given $$f$$ and the $$\psi_{j}$$ s there is a unique expansion in which the mode strengths are given by $x_{j}=\langle f, \psi j\rangle /\left\langle\psi_{j}, \psi_{j}\right\rangle$ An expansion of $$X$$ in this form is called a polynomial chaos expansion. Since there are many possible functions $$f$$ for given $$X$$ and $$\Xi$$ distributions, there are many possible expansions of a given $$X$$ using a given germ. They will differ in the mode strengths.

In practice we truncate the expansions to a finite number $$p$$ of terms, $X_{p}=f_{p}(\Xi)=\sum_{j=0}^{p} x_{j} \psi_{j}(\Xi).$

It is not clear immediately, but this gives us a tool to track propagation of error through a model.

There is a veritable zoo of bases to consider.

Here associated with means orthogonal with respect to.

## 2 “Generalized” chaos expansion

Wikipedia credits Xiu (2010) with the particular generalisation which apparently got naming rights for generalized chaos expansion, in the teeth of my private campaign for a moratorium on naming anything generalized [whatever]. I think it is about expanding the list of acceptable polynomial bases? TBC.

## 3 Arbitrary chaos expansion

Learnable sparse basis-style chaos expansions. See .

As far as I can tell from brusque mentions about the place, these methods construct polynomial basis expansions weighted by the empirical distribution of observationss. That is, instead of taking a given germ, we use an empirical estimate of a germ and calculate a basis over it by the Gram-Schmidt procedure.

## 4 References

Alexanderian. 2015. arXiv:1509.07526 [Math].
Alpay, and Kipnis. 2015. Numerical Functional Analysis and Optimization.
Calatayud Gregori, Chen-Charpentier, Cortés López, et al. 2019. Symmetry.
Cameron, and Martin. 1944. The Annals of Mathematics.
Franceschini, and Giardinà. 2017. arXiv:1701.09115 [Math-Ph].
Ghanem, Roger, and Red-Horse. 2017. In Handbook of Uncertainty Quantification.
Ghanem, Roger, and Spanos. 1990. Journal of Applied Mechanics.
Ghanem, Roger G., and Spanos. 2003. Stochastic Finite Elements: A Spectral Approach.
Gratiet, Marelli, and Sudret. 2016. In Handbook of Uncertainty Quantification.
Kim, and Braatz. 2013. International Journal of Control.
Kim, Shen, Nagy, et al. 2013. IEEE Control Systems Magazine.
Lei, Li, Gao, et al. 2018.
Levajkovic, and Selesi. 2011. Publications de l’Institut Mathematique.
Noorshams, and Wainwright. 2013. “Belief Propagation for Continuous State Spaces: Stochastic Message-Passing with Quantitative Guarantees.” The Journal of Machine Learning Research.
Nualart, and Schoutens. 2000. Stochastic Processes and Their Applications.
O’Hagan. 2013. “Polynomial Chaos: A Tutorial and Critique from a Statistician’s Perspective.”
Oladyshkin, and Nowak. 2012. Reliability Engineering & System Safety.
Rahman. 2017. arXiv:1704.07912 [Math].
Schoutens. 2000. Stochastic Processes and Orthogonal Polynomials. Lecture Notes in Statistics.
———. 2001. Journal of Mathematical Analysis and Applications.
Wan, and Karniadakis. 2006. SIAM Journal on Scientific Computing.
Wiener. 1938. American Journal of Mathematics.
Witteveen, and Bijl. 2006. In 44th AIAA Aerospace Sciences Meeting and Exhibit.
Xiu, and Hesthaven. 2005. SIAM Journal on Scientific Computing.
Xiu, and Karniadakis. 2002. SIAM Journal on Scientific Computing.
Zhang, Lu, Guo, et al. 2019. Journal of Computational Physics.
Zheng, Wan, and Karniadakis. 2015. Applied Numerical Mathematics.