As seen in chaos expansions kind-of, and used in low rank kriging, and assumed in PCA and implied by many basis decompositions etc.

Suppose we have a collection \(\{\varphi_n\}\) of real valued functions on our index space \(T\), and a collection \(\{\xi_n\}\) of uncorrelated random variables. Now we define the random process \[ f(t)=\sum_{n=1}^{\infty} \xi_{n} \varphi_{n}(t). \] We might care about the first two moments of \(f,\) i.e. \[ \mathbb{E}\{f(s) f(t)\}=\sum_{n=1}^{\infty} \sigma_{n}^{2} \varphi_{n}(s) \varphi_{n}(t) \] and variance function \[ \mathbb{E}\left\{f^{2}(t)\right\}=\sum_{n=1}^{\infty} \sigma_{n}^{2} \varphi_{n}^{2}(t) \]

Now suppose that we have a stochastic process where the index \(T\) is a compact domain in \(\mathbb{R}^{N}\). The corresponding expansion of \(f\) in the above form is known as the Karhunen-Loève expansion. Suppose that \(f\) has covariance function \(K\) and define an operator \(\mathcal{K}\), taking the space of square integrable functions on \(T\) to itself, by \[ (\mathcal{K} \psi)(t)=\int_{T} K(s, t) \psi(s) d s \] Suppose that \(\lambda_{1} \geq \lambda_{2} \geq \ldots\), and \(\psi_{1}, \psi_{2}, \ldots\), are, respectively, the (ordered) eigenvalues and normalised eigenfunctions of the operator. That is, the \(\lambda_{n}\) and \(\psi_{n}\) solve the integral equation \[ \int_{T} K(s, t) \psi(s) d s=\lambda \psi(t) \] with the normalisation \[ \int_{T} \psi_{n}(t) \psi_{m}(t) d t=\left\{\begin{array}{ll} 1 & n=m \\ 0 & n \neq m \end{array}\right. \] These eigenfunctions lead to a natural expansion of \(K\), known as Mercer’s Theorem, which states that \[ K(s, t)=\sum_{n=1}^{\infty} \lambda_{n} \psi_{n}(s) \psi_{n}(t) \] where the series converges absolutely and uniformly on \(T \times T\). The Karhunen-Loève expansion of \(f\) is obtained by setting \(\varphi_{n}=\) \(\lambda_{n}^{\frac{1}{2}} \psi_{n},\) so that \[ f_{t}=\sum_{n=1}^{\infty} \lambda_{n}^{\frac{1}{2}} \xi_{n} \psi_{n}(t). \] Now, when does such an expansion exist?

Because I look at signals on unbounded domains a lot, the case where a countable basis cannot be assumed to exist seems important.

## Gaussian

An extra-fun case, when this construction induces a Gaussian process because the \(\xi_{n}\) are Gaussian. Very handy in Gaussian process regression.

## References

*The Geometry of Random Fields*. SIAM ed. Philadelphia: Society for Industrial and Applied Mathematics.

*Random Fields and Geometry*. Springer Monographs in Mathematics 115. New York: Springer.

*Applications of Random Fields and Geometry Draft*.

*arXiv:1509.07526 [Math]*, October.

*Journal of Multivariate Analysis*120 (September): 1–17.

*Symmetry*11 (1): 43.

*Stochastic Finite Elements: A Spectral Approach*. New York, NY: Courier Corporation.

*Handbook of Uncertainty Quantification*, edited by Roger Ghanem, David Higdon, and Houman Owhadi, 521–51. Cham: Springer International Publishing.

*Journal of Applied Mechanics*57 (1): 197–202.

*Handbook of Uncertainty Quantification*, edited by Roger Ghanem, David Higdon, and Houman Owhadi, 1–37. Cham: Springer International Publishing.

*Inference for functional data with applications*. Vol. 200. Springer series in statistics. New York: Springer.

^{†}.”

*International Journal of Control*86 (8): 1324–37.

*IEEE Control Systems Magazine*33 (5): 58–67.

*arXiv:1411.4681 [Math, Stat]*, November.

*AStA Advances in Statistical Analysis*98 (2): 121–42.

*Spline Models for Observational Data*. SIAM.

*Numerical Methods for Stochastic Computations: A Spectral Method Approach*. USA: Princeton University Press.

*SIAM Journal on Scientific Computing*27 (3): 1118–39.

*Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Contributions to Probability Theory*, January, 593–622.

## No comments yet. Why not leave one?