Bayesian model calibration

April 11, 2017 — June 1, 2023

functional analysis
how do science
model selection
optimization
statmech
surrogate
Figure 1

AFAICT this is a fancy word for parameter estimation from simulation-heavy communities. Distinct from calibration for probabilistic predictions.

Closely related to AutoML, in that surrogate optimisation is a popular tool for such, and adaptive design of experiment.

1 Surrogate optimisation

Classic GP surrogate optimisation is a popular tool for model calibration, see Kennedy and O’Hagan (2001) for a classic example. More recent: Plumlee (2017).

2 MMD

See Dellaporta et al. (2022) for the application of maximum mean discrepancy to the problem of model calibration.

3 References

Bayarri, Maria J, Berger, Paulo, et al. 2007. “A Framework for Validation of Computer Models.” Technometrics.
Bayarri, M. J., Walsh, Berger, et al. 2007. “Computer Model Validation with Functional Output.” The Annals of Statistics.
Cockayne, and Duncan. 2020. “Probabilistic Gradients for Fast Calibration of Differential Equation Models.”
Dellaporta, Knoblauch, Damoulas, et al. 2022. “Robust Bayesian Inference for Simulator-Based Models via the MMD Posterior Bootstrap.” arXiv:2202.04744 [Cs, Stat].
Doherty. 2015. Calibration and uncertainty analysis for complex environmental models.
Dunbar, Duncan, Stuart, et al. 2022. “Ensemble Inference Methods for Models With Noisy and Expensive Likelihoods.” SIAM Journal on Applied Dynamical Systems.
Higdon, Gattiker, Williams, et al. 2008. “Computer Model Calibration Using High-Dimensional Output.” Journal of the American Statistical Association.
Huang, Li, Macheret, et al. 2020. “A Tutorial on Calibration Measurements and Calibration Models for Clinical Prediction Models.” Journal of the American Medical Informatics Association : JAMIA.
Izmailov, Maddox, Kirichenko, et al. 2020. “Subspace Inference for Bayesian Deep Learning.” In Proceedings of The 35th Uncertainty in Artificial Intelligence Conference.
Kennedy, and O’Hagan. 2001. “Bayesian Calibration of Computer Models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology).
Koermer, Loda, Noble, et al. 2023. “Active Learning for Simulator Calibration.”
Laloy, and Jacques. 2019. “Emulation of CPU-Demanding Reactive Transport Models: A Comparison of Gaussian Processes, Polynomial Chaos Expansion, and Deep Neural Networks.” Computational Geosciences.
Madan. 2014. “Recovering Statistical Theory in the Context of Model Calibrations.” Journal of Financial Econometrics.
McInerney, Thyer, Kavetski, et al. 2018. “A Simplified Approach to Produce Probabilistic Hydrological Model Predictions.” Environmental Modelling & Software.
O’Hagan. 1978. “Curve Fitting and Optimal Design for Prediction.” Journal of the Royal Statistical Society: Series B (Methodological).
Oakley, and Youngman. 2017. “Calibration of Stochastic Computer Simulators Using Likelihood Emulation.” Technometrics.
Perdikaris, and Karniadakis. 2016. “Model inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond.” Journal of the Royal Society, Interface.
Pleiss, Raghavan, Wu, et al. 2017. “On Fairness and Calibration.” In Advances In Neural Information Processing Systems.
Plumlee. 2017. “Bayesian Calibration of Inexact Computer Models.” Journal of the American Statistical Association.
Regis, and Shoemaker. 2013. “Combining Radial Basis Function Surrogates and Dynamic Coordinate Search in High-Dimensional Expensive Black-Box Optimization.” Engineering Optimization.
Sacks, Schiller, and Welch. 1989. “Designs for Computer Experiments.” Technometrics.
Sacks, Welch, Mitchell, et al. 1989. “Design and Analysis of Computer Experiments.” Statistical Science.
Thiagarajan, Venkatesh, Anirudh, et al. 2020. “Designing Accurate Emulators for Scientific Processes Using Calibration-Driven Deep Models.” Nature Communications.
Tonkin, and Doherty. 2009. “Calibration-Constrained Monte Carlo Analysis of Highly Parameterized Models Using Subspace Techniques.” Water Resources Research.