Learning Gaussian processes which map functions to functions

December 7, 2020 — February 25, 2022

Gaussian
generative
geometry
Hilbert space
how do science
kernel tricks
machine learning
PDEs
physics
regression
spatial
stochastic processes
time series

In which I discover how to learn operators via GPs. I suspect a lot of things break; What is a usable gaussian distribution over a mapping between functions?

Figure 1

It might be handy here to revisit the notation for Bayesian nonparametrics, since we don’t get the same kind of setup as when the distributions in question are finitely parameterised. TBC

1 Universal Kriging

Does universal kriging fit in this notebook? (Menafoglio, Secchi, and Rosa 2013) In this setting our observations are function-valued and we wish to spatially interpolate them. TBC. Keyword: Hilbert-Kriging. See Júlio Hoffimann’s Hilbert-Kriging lecture.

2 Hilbert-space valued GPs

TBC

3 References

Bakka, Rue, Fuglstad, et al. 2018. “Spatial Modeling with R-INLA: A Review.” WIREs Computational Statistics.
Bolin. 2016. Models and Methods for Random Fields in Spatial Statistics with Computational Efficiency from Markov Properties.
Brault, d’Alché-Buc, and Heinonen. 2016. “Random Fourier Features for Operator-Valued Kernels.” In Proceedings of The 8th Asian Conference on Machine Learning.
Brault, Lim, and d’Alché-Buc. n.d. “Scaling up Vector Autoregressive Models With Operator-Valued Random Fourier Features.”
Brouard, Szafranski, and D’Alché-Buc. 2016. “Input Output Kernel Regression: Supervised and Semi-Supervised Structured Output Prediction with Operator-Valued Kernels.” The Journal of Machine Learning Research.
Davison, and Ortiz. 2019. “FutureMapping 2: Gaussian Belief Propagation for Spatial AI.” arXiv:1910.14139 [Cs].
Dutordoir, Saul, Ghahramani, et al. 2022. “Neural Diffusion Processes.”
Gahungu, Lanyon, Álvarez, et al. 2022. “Adjoint-Aided Inference of Gaussian Process Driven Differential Equations.” In.
Heinonen, and d’Alché-Buc. 2014. “Learning Nonparametric Differential Equations with Operator-Valued Kernels and Gradient Matching.” arXiv:1411.5172 [Cs, Stat].
Hennig, Osborne, and Girolami. 2015. “Probabilistic Numerics and Uncertainty in Computations.” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.
Hildeman, Bolin, and Rychlik. 2019. “Joint Spatial Modeling of Significant Wave Height and Wave Period Using the SPDE Approach.” arXiv:1906.00286 [Stat].
Hu, and Steinsland. 2016. “Spatial Modeling with System of Stochastic Partial Differential Equations.” WIREs Computational Statistics.
Kadri, Duflos, Preux, et al. 2016. “Operator-Valued Kernels for Learning from Functional Response Data.” The Journal of Machine Learning Research.
Kadri, Rakotomamonjy, Preux, et al. 2012. “Multiple Operator-Valued Kernel Learning.” Advances in Neural Information Processing Systems.
Lian. 2007. “Nonlinear Functional Models for Functional Responses in Reproducing Kernel Hilbert Spaces.” Canadian Journal of Statistics.
Lim, d’Alché-Buc, Auliac, et al. 2015. “Operator-Valued Kernel-Based Vector Autoregressive Models for Network Inference.” Machine Learning.
Long, Wang, Krishnapriyan, et al. 2022. “AutoIP: A United Framework to Integrate Physics into Gaussian Processes.”
Menafoglio, Secchi, and Rosa. 2013. “A Universal Kriging Predictor for Spatially Dependent Functional Data of a Hilbert Space.” Electronic Journal of Statistics.
Minh. 2022. “Finite Sample Approximations of Exact and Entropic Wasserstein Distances Between Covariance Operators and Gaussian Processes.” SIAM/ASA Journal on Uncertainty Quantification.
Phillips, Seror, Hutchinson, et al. 2022. “Spectral Diffusion Processes.” In.
Saha, and Balamurugan. 2020. “Learning with Operator-Valued Kernels in Reproducing Kernel Krein Spaces.” In Advances in Neural Information Processing Systems.
Zammit-Mangion, and Cressie. 2021. “FRK: An R Package for Spatial and Spatio-Temporal Prediction with Large Datasets.” Journal of Statistical Software.
Zhang, Xu, and Zhang. 2012. “Refinement of Operator-Valued Reproducing Kernels.” The Journal of Machine Learning Research.