Defining dynamics via Gaussian processes

September 18, 2019 — September 18, 2019

Bayes
dynamical systems
Gaussian
generative
Hilbert space
kernel tricks
linear algebra
probability
sciml
signal processing
state space models
statistics
time series

Two classic flavours together, Gaussian Processes and dynamical_systems, where the dynamics are modelled by a Gaussian process.

Figure 1

Here we use Gaussian processes to define the dynamics, particularly to learn nonparametric transition, observation, or state densities. This is what (Turner, Deisenroth, and Rasmussen 2010; Frigola, Chen, and Rasmussen 2014; Frigola et al. 2013; Eleftheriadis et al. 2017) do.

This is distinct from calculating a Gaussian process posterior via a state filter, which is another way you can combine the concepts of dynamics and Gaussian process.

c.f. Variational filters and particle filters.

Possibly the same, recurrent Gaussian Processes? 🏗 (Mattos et al. 2016, 2017; Föll et al. 2017).

1 References

Eleftheriadis, Nicholson, Deisenroth, et al. 2017. “Identification of Gaussian Process State Space Models.” In Advances in Neural Information Processing Systems 30.
Föll, Haasdonk, Hanselmann, et al. 2017. “Deep Recurrent Gaussian Process with Variational Sparse Spectrum Approximation.” arXiv:1711.00799 [Stat].
Frigola, Chen, and Rasmussen. 2014. “Variational Gaussian Process State-Space Models.” In Advances in Neural Information Processing Systems 27.
Frigola, Lindsten, Schön, et al. 2013. “Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC.” In Advances in Neural Information Processing Systems 26.
Huber. 2014. “Recursive Gaussian Process: On-Line Regression and Learning.” Pattern Recognition Letters.
Mattos, Dai, Damianou, et al. 2016. “Recurrent Gaussian Processes.” In Proceedings of ICLR.
Mattos, Dai, Damianou, et al. 2017. “Deep Recurrent Gaussian Processes for Outlier-Robust System Identification.” Journal of Process Control, DYCOPS-CAB 2016,.
Mohammadi, Challenor, and Goodfellow. 2021. “Emulating Computationally Expensive Dynamical Simulators Using Gaussian Processes.” arXiv:2104.14987 [Stat].
Nickisch, Solin, and Grigorevskiy. 2018. “State Space Gaussian Processes with Non-Gaussian Likelihood.” In International Conference on Machine Learning.
Quiñonero-Candela, Girard, Larsen, et al. 2003. “Propagation of Uncertainty in Bayesian Kernel Models - Application to Multiple-Step Ahead Forecasting.” In 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP ’03).
Turner, Deisenroth, and Rasmussen. 2010. “State-Space Inference and Learning with Gaussian Processes.” In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics.
Wilkinson, Chang, Andersen, et al. 2020. “State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes.” In ICML.