Probabilistic numerics

Probabilistic Numerics claims:

Probabilistic numerics (PN) aims to quantify uncertainty arising from intractable or incomplete numerical computation and from stochastic input. This new paradigm treats a numerical problem as one of statistical inference instead. The probabilistic viewpoint provides a principled way to encode structural knowledge about a problem. By giving an explicit role to uncertainty from all sources, in particular from the computation itself, PN gives rise to new applications beyond the scope of classical methods.

Typical numerical tasks to which PN may be applied include optimization, integration, the solution of ordinary and partial differential equations, and the basic tasks of linear algebra, e.g. solution of linear systems and eigenvalue problems.

As well as offering an enriched reinterpretation of classical methods, the PN approach has several concrete practical points of value. The probabilistic interpretation of computation

  • allows to build customized methods for specific problems with bespoke priors
  • formalizes the design of adaptive methods using tools from decision theory
  • provides a way of setting parameters of numerical methods via the Bayesian formalism
  • expedites the solution of mutually related problems of similar type
  • naturally incorporates sources of stochasticity in the computation
  • can give structural uncertainty via a probability measure compared to an error estimate

and finally it offers a principled approach of including numerical error in the propagation of uncertainty through chains of computations.


Connection to sparse GPs?


Herding: Chai et al. (2019);WellingHerding2009.

Connection to ensemble Kalman methods?


Chai, Henry, Jean-Francois Ton, Roman Garnett, and Michael A. Osborne. 2019. Automated Model Selection with Bayesian Quadrature.” arXiv.
Hennig, Philipp, Ilse C.F. Ipsen, Maren Mahsereci, and Tim Sullivan. 2022. Probabilistic Numerical Methods - From Theory to Implementation (Dagstuhl Seminar 21432).” Edited by Philipp Hennig, Ilse C.F. Ipsen, Maren Mahsereci, and Tim Sullivan. Dagstuhl Reports 11 (9): 102–19.
Hennig, Philipp, Michael A. Osborne, and Mark Girolami. 2015. Probabilistic Numerics and Uncertainty in Computations.” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 471 (2179): 20150142.
Hennig, Philipp, Michael A. Osborne, and Hans P. Kersting. 2022. Probabilistic Numerics: Computation as Machine Learning. 1st edition. Cambridge New York, NY Melbourne New Delhi Singapore: Cambridge University Press.
Huszár, Ferenc, and David Duvenaud. 2016. Optimally-Weighted Herding Is Bayesian Quadrature.” arXiv.
O’Hagan, A. 1991. Bayes–Hermite Quadrature.” Journal of Statistical Planning and Inference 29 (3): 245–60.
Song, Le, Xinhua Zhang, Alex Smola, Arthur Gretton, and Bernhard Schölkopf. 2008. Tailoring Density Estimation via Reproducing Kernel Moment Matching.” In Proceedings of the 25th International Conference on Machine Learning, 992–99. ICML ’08. New York, NY, USA: Association for Computing Machinery.
Welling, Max. 2009. Herding Dynamical Weights to Learn.” In Proceedings of the 26th Annual International Conference on Machine Learning, 1121–28. ICML ’09. New York, NY, USA: Association for Computing Machinery.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.