Bayesian model calibration



AFAICT this is a fancy word for parameter estimation from simulation-heavy communities. Distinct from calibration for prbabilistic predictions.

Closely related to AutoML, in that surrogate optimisation is a popular tool for such, and adaptive design of experiment.

Surrogate optimisation

Classic GP surrogate optimisation is a popular tool for model calibration, see Kennedy and O’Hagan (2001) for a classic example. More recent: Plumlee (2017).

MMD

See Dellaporta et al. (2022) for the application of maximum mean discrepancy to the problem of model calibration.

References

Bayarri, M. J., D. Walsh, J. O. Berger, J. Cafeo, G. Garcia-Donato, F. Liu, J. Palomo, R. J. Parthasarathy, R. Paulo, and J. Sacks. 2007. β€œComputer Model Validation with Functional Output.” The Annals of Statistics 35 (5): 1874–1906.
Bayarri, Maria J, James O Berger, Rui Paulo, Jerry Sacks, John A Cafeo, James Cavendish, Chin-Hsu Lin, and Jian Tu. 2007. β€œA Framework for Validation of Computer Models.” Technometrics 49 (2): 138–54.
Cockayne, Jon, and Andrew B. Duncan. 2020. β€œProbabilistic Gradients for Fast Calibration of Differential Equation Models,” September.
Dellaporta, Charita, Jeremias Knoblauch, Theodoros Damoulas, and FranΓ§ois-Xavier Briol. 2022. β€œRobust Bayesian Inference for Simulator-Based Models via the MMD Posterior Bootstrap.” arXiv:2202.04744 [Cs, Stat], February.
Doherty, John. 2015. Calibration and uncertainty analysis for complex environmental models.
Dunbar, Oliver R. A., Andrew B. Duncan, Andrew M. Stuart, and Marie-Therese Wolfram. 2022. β€œEnsemble Inference Methods for Models With Noisy and Expensive Likelihoods.” SIAM Journal on Applied Dynamical Systems 21 (2): 1539–72.
Higdon, Dave, James Gattiker, Brian Williams, and Maria Rightley. 2008. β€œComputer Model Calibration Using High-Dimensional Output.” Journal of the American Statistical Association 103 (482): 570–83.
Huang, Yingxiang, Wentao Li, Fima Macheret, Rodney A Gabriel, and Lucila Ohno-Machado. 2020. β€œA Tutorial on Calibration Measurements and Calibration Models for Clinical Prediction Models.” Journal of the American Medical Informatics Association : JAMIA 27 (4): 621–33.
Izmailov, Pavel, Wesley J. Maddox, Polina Kirichenko, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. 2020. β€œSubspace Inference for Bayesian Deep Learning.” In Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, 1169–79. PMLR.
Kennedy, Marc C., and Anthony O’Hagan. 2001. β€œBayesian Calibration of Computer Models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63 (3): 425–64.
Koermer, Scott, Justin Loda, Aaron Noble, and Robert B. Gramacy. 2023. β€œActive Learning for Simulator Calibration.” arXiv.
Laloy, Eric, and Diederik Jacques. 2019. β€œEmulation of CPU-Demanding Reactive Transport Models: A Comparison of Gaussian Processes, Polynomial Chaos Expansion, and Deep Neural Networks.” Computational Geosciences 23 (5): 1193–1215.
Madan, Dilip B. 2014. β€œRecovering Statistical Theory in the Context of Model Calibrations.” Journal of Financial Econometrics 13 (2): nbu020.
McInerney, David, Mark Thyer, Dmitri Kavetski, Bree Bennett, Julien Lerat, Matthew Gibbs, and George Kuczera. 2018. β€œA Simplified Approach to Produce Probabilistic Hydrological Model Predictions.” Environmental Modelling & Software 109 (November): 306–14.
O’Hagan, A. 1978. β€œCurve Fitting and Optimal Design for Prediction.” Journal of the Royal Statistical Society: Series B (Methodological) 40 (1): 1–24.
Oakley, Jeremy E., and Benjamin D. Youngman. 2017. β€œCalibration of Stochastic Computer Simulators Using Likelihood Emulation.” Technometrics 59 (1): 80–92.
Perdikaris, Paris, and George Em Karniadakis. 2016. β€œModel inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond.” Journal of the Royal Society, Interface 13 (118): 20151107.
Pleiss, Geoff, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q. Weinberger. 2017. β€œOn Fairness and Calibration.” In Advances In Neural Information Processing Systems.
Plumlee, Matthew. 2017. β€œBayesian Calibration of Inexact Computer Models.” Journal of the American Statistical Association 112 (519): 1274–85.
Regis, Rommel G., and Christine A. Shoemaker. 2013. β€œCombining Radial Basis Function Surrogates and Dynamic Coordinate Search in High-Dimensional Expensive Black-Box Optimization.” Engineering Optimization 45 (5): 529–55.
Sacks, Jerome, Susannah B. Schiller, and William J. Welch. 1989. β€œDesigns for Computer Experiments.” Technometrics 31 (1): 41–47.
Sacks, Jerome, William J. Welch, Toby J. Mitchell, and Henry P. Wynn. 1989. β€œDesign and Analysis of Computer Experiments.” Statistical Science 4 (4): 409–23.
Thiagarajan, Jayaraman J., Bindya Venkatesh, Rushil Anirudh, Peer-Timo Bremer, Jim Gaffney, Gemma Anderson, and Brian Spears. 2020. β€œDesigning Accurate Emulators for Scientific Processes Using Calibration-Driven Deep Models.” Nature Communications 11 (1): 5622.
Tonkin, Matthew, and John Doherty. 2009. β€œCalibration-Constrained Monte Carlo Analysis of Highly Parameterized Models Using Subspace Techniques.” Water Resources Research 45 (12).

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.