How can I simulate a Gaussian Processes with a given covariance? Handy in GP regression, especially GP functional regression and spatial statistics.

Historical overview in Liu et al. (2019).

## Lattice tricks

On lattices we can make some computational shortcuts. See GP simulation on lattices.

## Basis tricks

TBD

## Simulating from posterior GPs

Probably many tricks, but I know of pathwise GPs.

## References

Abrahamsen, P., V. Kvernelv, and D. Barker. 2018. “Simulation Of Gaussian Random Fields Using The Fast Fourier Transform (Fft).” In, 2018:1–14. European Association of Geoscientists & Engineers.

Alexanderian, Alen. 2015. “A Brief Note on the Karhunen-Loève Expansion.”

*arXiv:1509.07526 [Math]*, October.Bingham, N., and Tasmin Symons. 2022. “Gaussian Random Fields: With and Without Covariances.”

*Theory of Probability and Mathematical Statistics*106 (June): 27–40.Bolin, David. 2016.

*Models and Methods for Random Fields in Spatial Statistics with Computational Efficiency from Markov Properties*.Chan, Grace, and Andrew T.A. Wood. 1997. “Algorithm AS 312: An Algorithm for Simulating Stationary Gaussian Random Fields.”

*Journal of the Royal Statistical Society: Series C (Applied Statistics)*46 (1): 171–81.Chan, G., and A. T. A. Wood. 1999. “Simulation of Stationary Gaussian Vector Fields.”

*Statistics and Computing*9 (4): 265–68.Chilès, Jean-Paul, and Christian Lantuéjoul. 2005. “Prediction by Conditional Simulation: Models and Algorithms.” In

*Space, Structure and Randomness: Contributions in Honor of Georges Matheron in the Field of Geostatistics, Random Sets and Mathematical Morphology*, edited by Michel Bilodeau, Fernand Meyer, and Michel Schmitt, 39–68. Lecture Notes in Statistics. New York, NY: Springer.Choromanski, Krzysztof, and Vikas Sindhwani. 2016. “Recycling Randomness with Structure for Sublinear Time Kernel Expansions.”

*arXiv:1605.09049 [Cs, Stat]*, May.Cotter, S. L., G. O. Roberts, A. M. Stuart, and D. White. 2013. “MCMC Methods for Functions: Modifying Old Algorithms to Make Them Faster.”

*Statistical Science*28 (3): 424–46.Davies, Tilman M., and David Bryant. 2013. “On Circulant Embedding for Gaussian Random Fields in R.”

*Journal of Statistical Software*55 (9).Dietrich, C. R., and G. N. Newsam. 1993. “A Fast and Exact Method for Multidimensional Gaussian Stochastic Simulations.”

*Water Resources Research*29 (8): 2861–69.———. 1997. “Fast and Exact Simulation of Stationary Gaussian Processes Through Circulant Embedding of the Covariance Matrix.”

*SIAM Journal on Scientific Computing*18 (4): 1088–1107.Doucet, Arnaud. 2010. “A Note on Efficient Conditional Simulation of Gaussian Distributions.” University of British Columbia.

Durrande, Nicolas, Vincent Adam, Lucas Bordeaux, Stefanos Eleftheriadis, and James Hensman. 2019. “Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era.” In

*Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics*, 2780–89. PMLR.Ellis, Robert L., and David C. Lay. 1992. “Factorization of Finite Rank Hankel and Toeplitz Matrices.”

*Linear Algebra and Its Applications*173 (August): 19–38.Erhel, Jocelyne, Mestapha Oumouni, Géraldine Pichot, and Franck Schoefs. n.d. “Analysis of Continuous Spectral Method for Sampling Stationary Gaussian Random Fields,” 26.

Galy-Fajou, Théo, Valerio Perrone, and Manfred Opper. 2021. “Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation.”

*Entropy*23 (8): 990.Gilboa, E., Y. Saatçi, and J. P. Cunningham. 2015. “Scaling Multidimensional Inference for Structured Gaussian Processes.”

*IEEE Transactions on Pattern Analysis and Machine Intelligence*37 (2): 424–36.Graham, Ivan G., Frances Y. Kuo, Dirk Nuyens, Rob Scheichl, and Ian H. Sloan. 2017a. “Analysis of Circulant Embedding Methods for Sampling Stationary Random Fields.”

*arXiv:1710.00751 [Math]*, October.———. 2017b. “Circulant Embedding with QMC — Analysis for Elliptic PDE with Lognormal Coefficients.”

*arXiv:1710.09254 [Math]*, October.Gray, Robert M. 2006.

*Toeplitz and Circulant Matrices: A Review*. Vol. 2.Guinness, Joseph, and Montserrat Fuentes. 2016. “Circulant Embedding of Approximate Covariances for Inference From Gaussian Data on Large Lattices.”

*Journal of Computational and Graphical Statistics*26 (1): 88–97.Haran, Murali. 2011. “Gaussian Random Field Models for Spatial Data.” In

*Handbook of Markov Chain Monte Carlo*, edited by Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng. Vol. 20116022. Chapman and Hall/CRC.Heinig, Georg, and Karla Rost. 2011. “Fast Algorithms for Toeplitz and Hankel Matrices.”

*Linear Algebra and Its Applications*435 (1): 1–59.Hoffman, Yehuda, and Erez Ribak. 1991. “Constrained Realizations of Gaussian Fields-A Simple Algorithm.”

*The Astrophysical Journal*380: L5–8.Lang, Annika, and Jürgen Potthoff. 2011. “Fast Simulation of Gaussian Random Fields.”

*Monte Carlo Methods and Applications*17 (3).Latz, Jonas, Marvin Eisenberger, and Elisabeth Ullmann. 2019. “Fast Sampling of Parameterised Gaussian Random Fields.”

*Computer Methods in Applied Mechanics and Engineering*348 (May): 978–1012.Liu, Yang, Jingfa Li, Shuyu Sun, and Bo Yu. 2019. “Advances in Gaussian Random Field Generation: A Review.”

*Computational Geosciences*23 (5): 1011–47.Murray, Iain, Ryan Adams, and David MacKay. 2010. “Elliptical Slice Sampling.” In

*Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics*, 541–48. JMLR Workshop and Conference Proceedings.Powell, Catherine E. 2014. “Generating Realisations of Stationary Gaussian Random Fields by Circulant Embedding.”

*Matrix*2 (2): 1.Rue, Havard. 2001. “Fast Sampling of Gaussian Markov Random Fields.”

*Journal of the Royal Statistical Society. Series B (Statistical Methodology)*63 (2): 325–38.Rue, Håvard, and Leonhard Held. 2005.

*Gaussian Markov Random Fields: Theory and Applications*. Monographs on Statistics and Applied Probability 104. Boca Raton: Chapman & Hall/CRC.Sidén, Per. 2020.

*Scalable Bayesian Spatial Analysis with Gaussian Markov Random Fields*. Vol. 15. Linköping Studies in Statistics. Linköping: Linköping University Electronic Press.Simpson, Daniel Peter. 2008. “Krylov Subspace Methods for Approximating Functions of Symmetric Positive Definite Matrices with Applications to Applied Statistics and Anomalous Diffusion.” Phd, Queensland University of Technology.

Simpson, Daniel P., Ian W. Turner, Christopher M. Strickland, and Anthony N. Pettitt. 2013. “Scalable Iterative Methods for Sampling from Massive Gaussian Random Vectors.” arXiv.

Stroud, Jonathan R., Michael L. Stein, and Shaun Lysen. 2017. “Bayesian and Maximum Likelihood Estimation for Gaussian Processes on an Incomplete Lattice.”

*Journal of Computational and Graphical Statistics*26 (1): 108–20.Teichmann, Jakob, and Karl-Gerald van den Boogaart. 2016. “Efficient Simulation of Stationary Multivariate Gaussian Random Fields with Given Cross-Covariance.”

*Applied Mathematics*7 (17): 2183–94.Whittle, P. 1954. “On Stationary Processes in the Plane.”

*Biometrika*41 (3/4): 434–49.Whittle, P. 1953a. “The Analysis of Multiple Stationary Time Series.”

*Journal of the Royal Statistical Society: Series B (Methodological)*15 (1): 125–39.———. 1953b. “Estimation and Information in Stationary Time Series.”

*Arkiv För Matematik*2 (5): 423–34.Whittle, Peter. 1952. “Some Results in Time Series Analysis.”

*Scandinavian Actuarial Journal*1952 (1-2): 48–60.Wilson, James T, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Marc Peter Deisenroth. 2021. “Pathwise Conditioning of Gaussian Processes.”

*Journal of Machine Learning Research*22 (105): 1–47.Yang, Linxiao, Jun Fang, Huiping Duan, Hongbin Li, and Bing Zeng. 2018. “Fast Low-Rank Bayesian Matrix Completion with Hierarchical Gaussian Prior Models.”

*IEEE Transactions on Signal Processing*66 (11): 2804–17.Ye, Ke, and Lek-Heng Lim. 2016. “Every Matrix Is a Product of Toeplitz Matrices.”

*Foundations of Computational Mathematics*16 (3): 577–98.
## No comments yet. Why not leave one?