Chaos expansions

Polynomial chaos, generalized polynomial chaos, arbitrary chaos etc

Placeholder, for a topic which has a slightly confusing name. To explore: Connection to/difference from other methods of keeping track of evolution of uncertainty in dynamical systems. C&C Gaussian process regression as used in Gratiet, Marelli, and Sudret (2016), functional data analysis etc.

This is not the same thing as chaos in the sense of the deterministic chaos made famous by dynamical systems theory and fractal t-shirts. Different term

Polynomial chaos expansion

Ground Zero, the original famous one. Wikipedia has an inscrutable introduction which I can make no sense of.

Popular introductions to the concept seem to be (R. Ghanem and Spanos 1990; R. G. Ghanem and Spanos 2003; Witteveen and Bijl 2006; Xiu and Karniadakis 2002; Kim et al. 2013). For my money, the most direct is O’Hagan (2013) which runs thusly: We define an inner product \[ \left\langle \psi, \phi\right\rangle=\int \psi(\xi) \phi(\xi) p_{\xi}(\xi) d \xi \] with respect to the probability density function \(p_{\xi}\) of some random variate \(\Xi\) which we call the germ.

Suppose we have chosen a functional basis comprising polynomials \(\psi_{0}=1, \psi_{1}, \psi_{2}, \ldots,\) where \(\psi_{j}\) is a polynomial of order \(j\) and where they satisfy the orthogonality condition that for all \(j \neq k\) \[ \left\langle\psi_{j}, \psi_{k}\right\rangle=0. \]

We write \[ X=f(\Xi)=\sum_{j=0}^{\infty} x_{j} \psi_{j}(\Xi) \] The combination of \(x_{j}\) (the mode strength) and \(\psi_{j}\) (the mode function) is called the \(j\) -th mode. By orthogonality, given \(f\) and the \(\psi_{j}\) s there is a unique expansion in which the mode strengths are given by \[ x_{j}=\langle f, \psi j\rangle /\left\langle\psi_{j}, \psi_{j}\right\rangle \] An expansion of \(X\) in this form is called a polynomial chaos expansion. Since there are many possible functions \(f\) for given \(X\) and \(\Xi\) distributions, there are many possible expansions of a given \(X\) using a given germ. They will differ in the mode strengths.

In practice we truncate the expansions to a finite number \(p\) of terms, \[ X_{p}=f_{p}(\Xi)=\sum_{j=0}^{p} x_{j} \psi_{j}(\Xi). \]

C&C Karhunen—Loève expansion.

It is not clear immediately, but this gives us a tool to track propagation of error through a model.

There is a veritable zoo of bases to consider.

Here associated with means orthogonal with respect to.

“Generalized” chaos expansion

Wikipedia credits Xiu (2010) with the particular generalisation which apparently got naming rights for generalized chaos expansion, in the teeth of my private campaign for a moratorium on naming anything generalized [whatever]. I think it is about expanding the list of acceptable polynomial bases? TBC.

Arbitrary chaos expansion

Learnable sparse basis-style chaos expansions. See (Witteveen and Bijl 2006; Zhang et al. 2019; Lei et al. 2018; Oladyshkin and Nowak 2012; Wan and Karniadakis 2006; Zheng, Wan, and Karniadakis 2015).

As far as I can tell from brusque mentions about the place, these methods construct polynomial basis expansions weighted by the empirical distribution of observationss. That is, instead of taking a given germ, we use an empirical estimate of a germ and calculate a basis over it by the Gram-Schmidt procedure.


Alexanderian, Alen. 2015. A Brief Note on the Karhunen-Loève Expansion.” arXiv:1509.07526 [Math], October.
Alpay, Daniel, and Alon Kipnis. 2015. Wiener Chaos Approach to Optimal Prediction.” Numerical Functional Analysis and Optimization 36 (10): 1286–306.
Calatayud Gregori, Julia, Benito M. Chen-Charpentier, Juan Carlos Cortés López, and Marc Jornet Sanz. 2019. Combining Polynomial Chaos Expansions and the Random Variable Transformation Technique to Approximate the Density Function of Stochastic Problems, Including Some Epidemiological Models.” Symmetry 11 (1): 43.
Cameron, R. H., and W. T. Martin. 1944. Transformations of Weiner Integrals Under Translations.” The Annals of Mathematics 45 (2): 386.
Franceschini, Chiara, and Cristian Giardinà. 2017. Stochastic Duality and Orthogonal Polynomials.” arXiv:1701.09115 [Math-Ph], January.
Ghanem, Roger G., and Pol D. Spanos. 2003. Stochastic Finite Elements: A Spectral Approach. New York, NY: Courier Corporation.
Ghanem, Roger, and John Red-Horse. 2017. Polynomial Chaos: Modeling, Estimation, and Approximation.” In Handbook of Uncertainty Quantification, edited by Roger Ghanem, David Higdon, and Houman Owhadi, 521–51. Cham: Springer International Publishing.
Ghanem, Roger, and P. D. Spanos. 1990. Polynomial Chaos in Stochastic Finite Elements.” Journal of Applied Mechanics 57 (1): 197–202.
Gratiet, Loïc Le, Stefano Marelli, and Bruno Sudret. 2016. Metamodel-Based Sensitivity Analysis: Polynomial Chaos Expansions and Gaussian Processes.” In Handbook of Uncertainty Quantification, edited by Roger Ghanem, David Higdon, and Houman Owhadi, 1–37. Cham: Springer International Publishing.
Kim, Kwang-Ki K., and Richard D. Braatz. 2013. Generalised Polynomial Chaos Expansion Approaches to Approximate Stochastic Model Predictive Control .” International Journal of Control 86 (8): 1324–37.
Kim, Kwang-Ki K., Dongying Erin Shen, Zoltan K. Nagy, and Richard D. Braatz. 2013. Wiener’s Polynomial Chaos for the Analysis and Control of Nonlinear Dynamical Systems with Probabilistic Uncertainties [Historical Perspectives].” IEEE Control Systems Magazine 33 (5): 58–67.
Lei, Huan, Jing Li, Peiyuan Gao, Panos Stinis, and Nathan Baker. 2018. A Data-Driven Framework for Sparsity-Enhanced Surrogates with Arbitrary Mutually Dependent Randomness,” April.
Levajkovic, Tijana, and Dora Selesi. 2011. Chaos Expansion Methods for Stochastic Differential Equations Involving the Malliavin Derivative, Part I.” Publications de l’Institut Mathematique 90 (104): 65–84.
Luo, Wuan. 2006. Wiener Chaos Expansion and Numerical Solutions of Stochastic Partial Differential Equations.” Phd, California Institute of Technology.
Noorshams, Nima, and Martin J. Wainwright. 2013. “Belief Propagation for Continuous State Spaces: Stochastic Message-Passing with Quantitative Guarantees.” The Journal of Machine Learning Research 14 (1): 2799–2835.
Nualart, David, and Wim Schoutens. 2000. Chaotic and Predictable Representations for Lévy Processes.” Stochastic Processes and Their Applications 90 (1): 109–22.
O’Hagan, Anthony. 2013. “Polynomial Chaos: A Tutorial and Critique from a Statistician’s Perspective,” 20.
Oladyshkin, S., and W. Nowak. 2012. Data-Driven Uncertainty Quantification Using the Arbitrary Polynomial Chaos Expansion.” Reliability Engineering & System Safety 106 (October): 179–90.
Rahman, Sharif. 2017. Wiener-Hermite Polynomial Expansion for Multivariate Gaussian Probability Measures.” arXiv:1704.07912 [Math], April.
Schoutens, Wim. 2000. Stochastic Processes and Orthogonal Polynomials. Lecture Notes in Statistics. New York: Springer-Verlag.
———. 2001. Orthogonal Polynomials in Stein’s Method.” Journal of Mathematical Analysis and Applications 253 (2): 515–31.
Wan, Xiaoliang, and George Em Karniadakis. 2006. Multi-Element Generalized Polynomial Chaos for Arbitrary Probability Measures.” SIAM Journal on Scientific Computing 28 (3): 901–28.
Wiener, Norbert. 1938. The Homogeneous Chaos.” American Journal of Mathematics 60 (4): 897.
Witteveen, Jeroen A. S., and Hester Bijl. 2006. Modeling Arbitrary Uncertainties Using Gram-Schmidt Polynomial Chaos.” In 44th AIAA Aerospace Sciences Meeting and Exhibit. American Institute of Aeronautics and Astronautics.
Xiu, Dongbin. 2010. Numerical Methods for Stochastic Computations: A Spectral Method Approach. USA: Princeton University Press.
Xiu, Dongbin, and Jan S. Hesthaven. 2005. High-Order Collocation Methods for Differential Equations with Random Inputs.” SIAM Journal on Scientific Computing 27 (3): 1118–39.
Xiu, Dongbin, and George Em Karniadakis. 2002. The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations.” SIAM Journal on Scientific Computing 24 (2): 619–44.
Zhang, Dongkun, Lu Lu, Ling Guo, and George Em Karniadakis. 2019. Quantifying Total Uncertainty in Physics-Informed Neural Networks for Solving Forward and Inverse Stochastic Problems.” Journal of Computational Physics 397 (November): 108850.
Zheng, Mengdi, Xiaoliang Wan, and George Em Karniadakis. 2015. Adaptive Multi-Element Polynomial Chaos with Discrete Measure: Algorithms and Application to SPDEs.” Applied Numerical Mathematics 90 (April): 91–110.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.