# Gaussian processes on lattices

October 30, 2019 — February 28, 2022

Bayes
dynamical systems
linear algebra
probability
signal processing
state space models
statistics
time series

$\renewcommand{\var}{\operatorname{Var}} \renewcommand{\dd}{\mathrm{d}} \renewcommand{\pd}{\partial} \renewcommand{\vv}[1]{\boldsymbol{#1}} \renewcommand{\mm}[1]{\mathrm{#1}} \renewcommand{\mmm}[1]{\mathrm{#1}} \renewcommand{\cc}[1]{\mathcal{#1}} \renewcommand{\oo}[1]{\operatorname{#1}} \renewcommand{\gvn}{\mid} \renewcommand{\II}[1]{\mathbb{I}\{#1\}} \renewcommand{\inner}[2]{\langle #1,#2\rangle} \renewcommand{\Inner}[2]{\left\langle #1,#2\right\rangle} \renewcommand{\norm}[1]{\| #1\|} \renewcommand{\Norm}[1]{\|\langle #1\right\|} \renewcommand{\argmax}{\operatorname{arg max}} \renewcommand{\argmin}{\operatorname{arg min}} \renewcommand{\omp}{\mathop{\mathrm{OMP}}}$

Gaussian Processes with a stationary kernel are faster if you are working on a grid of points. I have not used this trick, but I understand it involves various simplifications arising from the structure of Gram matrices which end up being Kronecker products of Toeplitz matrices under lexical ordering of the input points, which is long words to describe a fairly simple thing. The keyword to highlight this is Kronecker inference in the ML literature . Another keyword is circulant embedding, the approximation of a Toeplitz matrix by a circulant matrix, which apparently enables one to leverage fast Fourier transforms to calculate some quantities of interest, and some other nice linear algebra properties besides. That would imply that this method is has its origins in Whittle likelihoods , if I am not mistaken.

Lattice GP methods complements, perhaps, the trick of filtering Gaussian processes, which can also exploit structure lattice inputs, although the setup is different between these.

TBC.

## 2 In regression

OK, so what does this allow us to do with posterior inference in GPs? Apparently quite a lot. The KISS-GP method presumably leverages something similar. So do Saatçi (2012) and Flaxman et al. (2015).

Flaxman et al. (2015) deals with the lattice structure trick in a non-Gaussian likelihood setting using Laplace approximation.

TBC

## 4 References

Abrahamsen, Kvernelv, and Barker. 2018. In.
Alexanderian. 2015. arXiv:1509.07526 [Math].
Chan, Grace, and Wood. 1997. Journal of the Royal Statistical Society: Series C (Applied Statistics).
Chan, G., and Wood. 1999. Statistics and Computing.
Charlier, Feydy, Glaunès, et al. 2021. Journal of Machine Learning Research.
Choromanski, and Sindhwani. 2016. arXiv:1605.09049 [Cs, Stat].
Choudhuri, Ghosal, and Roy. 2004. Biometrika.
Cotter, Roberts, Stuart, et al. 2013. Statistical Science.
Davies, and Bryant. 2013. Journal of Statistical Software.
Dietrich, and Newsam. 1993. Water Resources Research.
———. 1997. SIAM Journal on Scientific Computing.
Durrande, Adam, Bordeaux, et al. 2019. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics.
Ellis, and Lay. 1992. Linear Algebra and Its Applications.
Erhel, Oumouni, Pichot, et al. n.d. “Analysis of Continuous Spectral Method for Sampling Stationary Gaussian Random Fields.”
Flaxman, Wilson, Neill, et al. 2015. “Fast Kronecker Inference in Gaussian Processes with Non-Gaussian Likelihoods.” In.
Gilboa, Saatçi, and Cunningham. 2015. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Graham, Kuo, Nuyens, et al. 2017a. arXiv:1710.00751 [Math].
———, et al. 2017b. arXiv:1710.09254 [Math].
Gray. 2006. Toeplitz and Circulant Matrices: A Review.
Guinness, and Fuentes. 2016. Journal of Computational and Graphical Statistics.
Heinig, and Rost. 2011. Linear Algebra and Its Applications.
Kroese, and Botev. 2013. arXiv:1308.0399 [Stat].
Lang, and Potthoff. 2011. Monte Carlo Methods and Applications.
Loper, Blei, Cunningham, et al. 2021. Journal of Machine Learning Research.
Nowak, and Litvinenko. 2013. Mathematical Geosciences.
Pleiss, Gardner, Weinberger, et al. 2018. In.
Powell. 2014. “Generating Realisations of Stationary Gaussian Random Fields by Circulant Embedding.” Matrix.
Rue, Havard. 2001. Journal of the Royal Statistical Society. Series B (Statistical Methodology).
Rue, Håvard, and Held. 2005. Gaussian Markov Random Fields: Theory and Applications. Monographs on Statistics and Applied Probability 104.
Saatçi. 2012.
Saatçi, Turner, and Rasmussen. 2010. In Proceedings of the 27th International Conference on International Conference on Machine Learning. ICML’10.
Sigrist, Künsch, and Stahel. 2015a. Application/pdf. Journal of Statistical Software.
———. 2015b. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
Stroud, Stein, and Lysen. 2017. Journal of Computational and Graphical Statistics.
Teichmann, and van den Boogaart. 2016. Applied Mathematics.
Ubaru, Chen, and Saad. 2017. SIAM Journal on Matrix Analysis and Applications.
Whittle, Peter. 1952. Scandinavian Actuarial Journal.
Whittle, P. 1952. Biometrika.
———. 1953a. Journal of the Royal Statistical Society: Series B (Methodological).
———. 1953b. Arkiv För Matematik.
Whittle, P. 1954. “On Stationary Processes in the Plane.” Biometrika.
Wilson, James T, Borovitskiy, Terenin, et al. 2021. Journal of Machine Learning Research.
Wilson, Andrew Gordon, Dann, and Nickisch. 2015. arXiv:1511.01870 [Cs, Stat].
Wilson, Andrew Gordon, and Nickisch. 2015. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37. ICML’15.
Ye, and Lim. 2016. Foundations of Computational Mathematics.