Localized Gaussian processes

October 16, 2020 — October 26, 2020

graphical models
Hilbert space
kernel tricks
machine learning

Approximate covariance by nearby predictors.

Stein (2002):

When predicting the value of a stationary random field at a location x in some region in which one has a large number of observations, it may be difficult to compute the optimal predictor. One simple way to reduce the computational burden is to base the predictor only on those observations nearest to x. As long as the number of observations used in the predictor is sufficiently large, one might generally expect the best predictor based on these observations to be nearly optimal relative to the best predictor using all observations. Indeed, this phenomenon has been empirically observed in numerous circumstances and is known as the screening effect in the geostatistical literature. For linear predictors, when observations are on a regular grid, this work proves that there generally is a screening effect as the grid becomes increasingly dense. This result requires that, at high frequencies, the spectral density of the random field not decay faster than algebraically and not vary too quickly. Examples demonstrate that there may be no screening effect if these conditions on the spectral density are violated.

1 References

Fuhg, Marino, and Bouklas. 2022. Local Approximate Gaussian Process Regression for Data-Driven Constitutive Models: Development and Comparison with Neural Networks.” Computer Methods in Applied Mechanics and Engineering.
Gramacy. 2016. laGP: Large-Scale Spatial Modeling via Local Approximate Gaussian Processes in R.” Journal of Statistical Software.
Gramacy, and Apley. 2015. Local Gaussian Process Approximation for Large Computer Experiments.” Journal of Computational and Graphical Statistics.
Snelson, and Ghahramani. 2007. Local and Global Sparse Gaussian Process Approximations.” In Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics.
Stein. 2002. The Screening Effect in Kriging.” The Annals of Statistics.