Karhunen-Loève expansions

As seen in chaos expansions kind-of, and used in low rank kriging, and assumed in PCA and implied by many basis decompositions etc.

Suppose we have a collection \(\{\varphi_n\}\) of real valued functions on our index space \(T\), and a collection \(\{\xi_n\}\) of uncorrelated random variables. Now we define the random process \[ f(t)=\sum_{n=1}^{\infty} \xi_{n} \varphi_{n}(t). \] We might care about the first two moments of \(f,\) i.e. \[ \mathbb{E}\{f(s) f(t)\}=\sum_{n=1}^{\infty} \sigma_{n}^{2} \varphi_{n}(s) \varphi_{n}(t) \] and variance function \[ \mathbb{E}\left\{f^{2}(t)\right\}=\sum_{n=1}^{\infty} \sigma_{n}^{2} \varphi_{n}^{2}(t) \]

Now suppose that we have a stochastic process where the index \(T\) is a compact domain in \(\mathbb{R}^{N}\). The corresponding expansion of \(f\) in the above form is known as the Karhunen-Loève expansion. Suppose that \(f\) has covariance function \(K\) and define an operator \(\mathcal{K}\), taking the space of square integrable functions on \(T\) to itself, by \[ (\mathcal{K} \psi)(t)=\int_{T} K(s, t) \psi(s) d s \] Suppose that \(\lambda_{1} \geq \lambda_{2} \geq \ldots\), and \(\psi_{1}, \psi_{2}, \ldots\), are, respectively, the (ordered) eigenvalues and normalised eigenfunctions of the operator. That is, the \(\lambda_{n}\) and \(\psi_{n}\) solve the integral equation \[ \int_{T} K(s, t) \psi(s) d s=\lambda \psi(t) \] with the normalisation \[ \int_{T} \psi_{n}(t) \psi_{m}(t) d t=\left\{\begin{array}{ll} 1 & n=m \\ 0 & n \neq m \end{array}\right. \] These eigenfunctions lead to a natural expansion of \(K\), known as Mercer’s Theorem, which states that \[ K(s, t)=\sum_{n=1}^{\infty} \lambda_{n} \psi_{n}(s) \psi_{n}(t) \] where the series converges absolutely and uniformly on \(T \times T\). The Karhunen-Loève expansion of \(f\) is obtained by setting \(\varphi_{n}=\) \(\lambda_{n}^{\frac{1}{2}} \psi_{n},\) so that \[ f_{t}=\sum_{n=1}^{\infty} \lambda_{n}^{\frac{1}{2}} \xi_{n} \psi_{n}(t). \] Now, when does such an expansion exist?

Because I look at signals on unbounded domains a lot, the case where a countable basis cannot be assumed to exist seems important.