# Gaussian processes on lattices

October 30, 2019 — February 28, 2022

\[ \renewcommand{\var}{\operatorname{Var}} \renewcommand{\dd}{\mathrm{d}} \renewcommand{\pd}{\partial} \renewcommand{\vv}[1]{\boldsymbol{#1}} \renewcommand{\mm}[1]{\mathrm{#1}} \renewcommand{\mmm}[1]{\mathrm{#1}} \renewcommand{\cc}[1]{\mathcal{#1}} \renewcommand{\oo}[1]{\operatorname{#1}} \renewcommand{\gvn}{\mid} \renewcommand{\II}[1]{\mathbb{I}\{#1\}} \renewcommand{\inner}[2]{\langle #1,#2\rangle} \renewcommand{\Inner}[2]{\left\langle #1,#2\right\rangle} \renewcommand{\norm}[1]{\| #1\|} \renewcommand{\Norm}[1]{\|\langle #1\right\|} \renewcommand{\argmax}{\operatorname{arg max}} \renewcommand{\argmin}{\operatorname{arg min}} \renewcommand{\omp}{\mathop{\mathrm{OMP}}} \]

Gaussian Processes with a stationary kernel are faster if you are working on a grid of points. I have not used this trick, but I understand it involves various simplifications arising from the structure of Gram matrices which end up being Kronecker products of Toeplitz matrices under lexical ordering of the input points, which is long words to describe a fairly simple thing. The keyword to highlight this is *Kronecker inference* in the ML literature (e.g. Saatçi 2012; Flaxman et al. 2015; A. G. Wilson and Nickisch 2015). Another keyword is *circulant embedding*, the approximation of a Toeplitz matrix by a circulant matrix, which apparently enables one to leverage fast Fourier transforms to calculate some quantities of interest, and some other nice linear algebra properties besides. That would imply that this method is has its origins in Whittle likelihoods (P. Whittle 1953a; Peter Whittle 1952), if I am not mistaken.

Lattice GP methods complements, perhaps, the trick of filtering Gaussian processes, which can also exploit structure lattice inputs, although the setup is different between these.

TBC.

## 1 Simulating lattice Gaussian fields with the desired covariance structure

## 2 In regression

OK, so what does this allow us to do with posterior inference in GPs? Apparently quite a lot. The KISS-GP method (A. G. Wilson and Nickisch 2015) presumably leverages something similar. So do Saatçi (2012) and Flaxman et al. (2015).

Flaxman et al. (2015) deals with the lattice structure trick in a non-Gaussian likelihood setting using Laplace approximation.

TBC

## 3 Incoming

- Scaling multi-output Gaussian process models with exact inference
- Gaussian Process Latent Variable Model
- Cox Processes (w/ Pyro/GPyTorch Low-Level Interface)
- Latent Function Inference with Pyro + GPyTorch (Low-Level Interface)
- Exact DKL (Deep Kernel Learning) Regression w/ KISS-GP
- Exact GPs with Scalable (GPU) Inference — GPyTorch
- GP Regression with LOVE for Fast Predictive Variances and Sampling

## 4 References

*arXiv:1509.07526 [Math]*.

*Journal of the Royal Statistical Society: Series C (Applied Statistics)*.

*Statistics and Computing*.

*Journal of Machine Learning Research*.

*arXiv:1605.09049 [Cs, Stat]*.

*Biometrika*.

*Statistical Science*.

*Journal of Statistical Software*.

*Water Resources Research*.

*SIAM Journal on Scientific Computing*.

*Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics*.

*Linear Algebra and Its Applications*.

*IEEE Transactions on Pattern Analysis and Machine Intelligence*.

*arXiv:1710.00751 [Math]*.

*arXiv:1710.09254 [Math]*.

*Toeplitz and Circulant Matrices: A Review*.

*Journal of Computational and Graphical Statistics*.

*Linear Algebra and Its Applications*.

*arXiv:1308.0399 [Stat]*.

*Monte Carlo Methods and Applications*.

*arXiv:2003.05554 [Cs, Stat]*.

*Mathematical Geosciences*.

*Matrix*.

*Journal of the Royal Statistical Society. Series B (Statistical Methodology)*.

*Gaussian Markov Random Fields: Theory and Applications*. Monographs on Statistics and Applied Probability 104.

*Proceedings of the 27th International Conference on International Conference on Machine Learning*. ICML’10.

*Journal of Statistical Software*.

*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*.

*Journal of Computational and Graphical Statistics*.

*Applied Mathematics*.

*SIAM Journal on Matrix Analysis and Applications*.

*Scandinavian Actuarial Journal*.

*Biometrika*.

*Journal of the Royal Statistical Society: Series B (Methodological)*.

*Arkiv För Matematik*.

*Biometrika*.

*Journal of Machine Learning Research*.

*arXiv:1511.01870 [Cs, Stat]*.

*Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37*. ICML’15.

*Foundations of Computational Mathematics*.