How can I simulate a Gaussian Processes on a lattice with a given covariance?

The general (non-lattice) case is given in historical overview in Liu et al. (2019), but in this notebook we are interested in specialising a little. Following the introduction in Dietrich and Newsam (1993), let’s say we wish to generate a stationary Gaussian process \(Y(x)\) on a points \(\Omega\). \(\Omega=(x_0, x_1,\dots, x_m)\).

*Stationary* in this context means that the covariance function
\(r\) is translation-invariance and depend only on distance,
so that it may be given \(r(|x|)\).
Without loss of generality,
we assume that \(\bb E[Y(x)]=0\) and \(\var[Y(x)]=1\).

The problem then reduces to generating a vector \(\vv y=(Y(x_0), Y(x_1), \dots, Y(x_m) )\sim \mathcal{N}(0, R)\) where \(R\) has entries \(R[p,q]=r(|x_p-x_q|).\)

Note that if \(\bb \varepsilon\sim\mathcal{N}(0, I)\) is an \(m+1\)-dimensional normal random variable, and \(AA^T=R\), then \(\vv y=\mm A \vv \varepsilon\) has the required distribution.

## The circulant embedding trick

If we have additional structure, we can work more efficiently.

Suppose further that our points form a grid, \(\Omega=(x_0, x_0+h,\dots, x_0+mh)\); specifically, equally-spaced-points on a line.

We know that \(R\) has a Toeplitz structure. Moreover it is non-negative definite, with \(\vv x^t\mm R \vv x \geq 0\forall \vv x.\) (Why?) 🏗

Wilson et al. (2021) credits the following authors:

Well-known examples of this trend include banded and sparse matrices in the context of one-dimensional Gaussian processes and Gauss–Markov random fields [RueGaussian2005@LoperLineartime2021;Durrande et al. (2019)], as well as Kronecker and Toeplitz matrices when working with regularly-spaced grids (Dietrich and Newsam 1997; Grace Chan and Wood 1997).

## References

*arXiv:1509.07526 [Math]*, October.

*Models and Methods for Random Fields in Spatial Statistics with Computational Efficiency from Markov Properties*.

*Journal of the Royal Statistical Society: Series C (Applied Statistics)*46 (1): 171–81.

*Statistics and Computing*9 (4): 265–68.

*arXiv:1605.09049 [Cs, Stat]*, May.

*Statistical Science*28 (3): 424–46.

*Journal of Statistical Software*55 (9).

*Water Resources Research*29 (8): 2861–69.

*SIAM Journal on Scientific Computing*18 (4): 1088–1107.

*Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics*, 2780–89. PMLR.

*Linear Algebra and Its Applications*173 (August): 19–38.

*Entropy*23 (8): 990.

*IEEE Transactions on Pattern Analysis and Machine Intelligence*37 (2): 424–36.

*arXiv:1710.00751 [Math]*, October.

*arXiv:1710.09254 [Math]*, October.

*Toeplitz and Circulant Matrices: A Review*. Vol. 2.

*Journal of Computational and Graphical Statistics*26 (1): 88–97.

*Handbook of Markov Chain Monte Carlo*, edited by Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng. Vol. 20116022. Chapman and Hall/CRC.

*Linear Algebra and Its Applications*435 (1): 1–59.

*Monte Carlo Methods and Applications*17 (3).

*Computer Methods in Applied Mechanics and Engineering*348 (May): 978–1012.

*Computational Geosciences*23 (5): 1011–47.

*Matrix*2 (2): 1.

*Journal of the Royal Statistical Society. Series B (Statistical Methodology)*63 (2): 325–38.

*Gaussian Markov Random Fields: Theory and Applications*. Monographs on Statistics and Applied Probability 104. Boca Raton: Chapman & Hall/CRC.

*Scalable Bayesian Spatial Analysis with Gaussian Markov Random Fields*. Vol. 15. Linköping Studies in Statistics. Linköping: Linköping University Electronic Press.

*Journal of Computational and Graphical Statistics*26 (1): 108–20.

*Applied Mathematics*7 (17): 2183–94.

*Biometrika*41 (3/4): 434–49.

*Journal of the Royal Statistical Society: Series B (Methodological)*15 (1): 125–39.

*Arkiv För Matematik*2 (5): 423–34.

*Scandinavian Actuarial Journal*1952 (1-2): 48–60.

*Journal of Machine Learning Research*22 (105): 1–47.

*IEEE Transactions on Signal Processing*66 (11): 2804–17.

*Foundations of Computational Mathematics*16 (3): 577–98.

## No comments yet. Why not leave one?