How can I simulate a Gaussian Processes on a lattice with a given covariance?

The general (non-lattice) case is given in historical overview in Liu et al. (2019), but in this notebook we are interested in specialising a little. Following the introduction in Dietrich and Newsam (1993), letβs say we wish to generate a stationary Gaussian process \(Y(x)\) on a points \(\Omega\). \(\Omega=(x_0, x_1,\dots, x_m)\).

*Stationary* in this context means that the covariance function
\(r\) is translation-invariance and depend only on distance,
so that it may be given \(r(|x|)\).
Without loss of generality,
we assume that \(\bb E[Y(x)]=0\) and \(\var[Y(x)]=1\).

The problem then reduces to generating a vector \(\vv y=(Y(x_0), Y(x_1), \dots, Y(x_m) )\sim \mathcal{N}(0, R)\) where \(R\) has entries \(R[p,q]=r(|x_p-x_q|).\)

Note that if \(\bb \varepsilon\sim\mathcal{N}(0, I)\) is an \(m+1\)-dimensional normal random variable, and \(AA^T=R\), then \(\vv y=\mm A \vv \varepsilon\) has the required distribution.

## The circulant embedding trick

If we have additional structure, we can work more efficiently.

Suppose further that our points form a grid, \(\Omega=(x_0, x_0+h,\dots, x_0+mh)\); specifically, equally-spaced-points on a line.

We know that \(R\) has a Toeplitz structure. Moreover it is non-negative definite, with \(\vv x^t\mm R \vv x \geq 0\forall \vv x.\) (Why?) π

Wilson et al. (2021) credits the following authors:

Well-known examples of this trend include banded and sparse matrices in the context of one-dimensional Gaussian processes and GaussβMarkov random fields [RueGaussian2005@LoperLineartime2021;Durrande et al. (2019)], as well as Kronecker and Toeplitz matrices when working with regularly-spaced grids (Dietrich and Newsam 1997; Grace Chan and Wood 1997).

## References

*Journal of the Royal Statistical Society: Series C (Applied Statistics)*46 (1): 171β81.

*Statistics and Computing*9 (4): 265β68.

*Journal of Statistical Software*55 (9).

*Water Resources Research*29 (8): 2861β69.

*SIAM Journal on Scientific Computing*18 (4): 1088β1107.

*Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics*, 2780β89. PMLR.

*IEEE Transactions on Pattern Analysis and Machine Intelligence*37 (2): 424β36.

*arXiv:1710.00751 [Math]*, October.

*arXiv:1710.09254 [Math]*, October.

*Toeplitz and Circulant Matrices: A Review*. Vol. 2.

*Journal of Computational and Graphical Statistics*26 (1): 88β97.

*Handbook of Markov Chain Monte Carlo*, edited by Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng. Vol. 20116022. Chapman and Hall/CRC.

*Monte Carlo Methods and Applications*17 (3).

*Computational Geosciences*23 (5): 1011β47.

*Matrix*2 (2): 1.

*Journal of the Royal Statistical Society. Series B (Statistical Methodology)*63 (2): 325β38.

*Gaussian Markov Random Fields: Theory and Applications*. Monographs on Statistics and Applied Probability 104. Boca Raton: Chapman & Hall/CRC.

*Applied Mathematics*7 (17): 2183β94.

*Biometrika*41 (3/4): 434β49.

*Biometrika*39 (3-4): 309β18.

*Journal of the Royal Statistical Society: Series B (Methodological)*15 (1): 125β39.

*Arkiv FΓΆr Matematik*2 (5): 423β34.

*Scandinavian Actuarial Journal*1952 (1-2): 48β60.

*Journal of Machine Learning Research*22 (105): 1β47.

## No comments yet. Why not leave one?