See also M-open.
For now, consider Christian Robert’s brief intro:
A few thoughts (and many links to my blog entries!) about that meme that all models are wrong:
- While the hypothetical model is indeed almost invariably and irremediably wrong, it still makes sense to act in an efficient or coherent manner with respect to this model if this is the best one can do. The resulting inference produces an evaluation of the formal model that is the “closest” to the actual data generating model (if any);
- There exist Bayesian approaches that can do without the model, a most recent example being the papers by Bissiri et al. (with my comments) and by Watson and Holmes (which I discussed with Judith Rousseau);
- In a connected way, there exists a whole branch of Bayesian statistics dealing with M-open inference;
- And yet another direction I like a lot is the SafeBayes approach of Peter Grünwald, who takes into account model misspecification to replace the likelihood with a down-graded version expressed as a power of the original likelihood.
- The very recent Read Paper by Gelman and Hennig addresses this issue, albeit in a circumvoluted manner (and I added some comments on my blog). I presume you could gather material for a discussion from the entries about your question.
- In a sense, Bayesians should be the least concerned among statisticians and modellers about this aspect since the sampling model is to be taken as one of several prior assumptions and the outcome is conditional or relative to all those prior assumptions.
Baek, Youngsoo, Wilkins Aquino, and Sayan Mukherjee. 2023. “Generalized Bayes Approach to Inverse Problems with Model Misspecification.” Inverse Problems 39 (10): 105011.
Bissiri, P. G., C. C. Holmes, and S. G. Walker. 2016. “A General Framework for Updating Belief Distributions.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 78 (5): 1103–30.
Bochkina, Natalia. 2023. “Bernstein–von Mises Theorem and Misspecified Models: A Review.” In Foundations of Modern Statistics, edited by Denis Belomestny, Cristina Butucea, Enno Mammen, Eric Moulines, Markus Reiß, and Vladimir V. Ulyanov, 355–80. Springer Proceedings in Mathematics & Statistics. Cham: Springer International Publishing.
Gill, Jeff, and Gary King. 2004. “What to Do When Your Hessian Is Not Invertible: Alternatives to Model Respecification in Nonlinear Estimation.” Sociological Methods & Research 33 (1): 54–87.
Grünwald, Peter, and Thijs van Ommen. 2017. “Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It.” Bayesian Analysis 12 (4): 1069–1103.
Kleijn, B. J. K., and A. W van der Vaart. 2006. “Misspecification in Infinite-Dimensional Bayesian Statistics.” The Annals of Statistics 34 (2): 837–77.
Kleijn, B. J. K., and A. W. van der Vaart. 2012. “The Bernstein-Von-Mises Theorem Under Misspecification.” Electronic Journal of Statistics 6 (none): 354–81.
Loecher, Markus. 2021. “The Perils of Misspecified Priors and Optional Stopping in Multi-Armed Bandits.” Frontiers in Artificial Intelligence 4 (July): 715690.
Masegosa, Andrés R. 2020. “Learning Under Model Misspecification: Applications to Variational and Ensemble Methods.” In Proceedings of the 34th International Conference on Neural Information Processing Systems, 5479–91. NIPS’20. Red Hook, NY, USA: Curran Associates Inc.
Medina, Marco Avella, José Luis Montiel Olea, Cynthia Rush, and Amilcar Velez. 2021. “On the Robustness to Misspecification of \(\alpha\)-Posteriors and Their Variational Approximations.”
Müller, Ulrich K. 2013. “Risk of Bayesian Inference in Misspecified Models, and the Sandwich Covariance Matrix.” Econometrica 81 (5): 1805–49.
Nott, David J., Christopher Drovandi, and David T. Frazier. 2023. “Bayesian Inference for Misspecified Generative Models.” arXiv.
Pati, Debdeep, Anirban Bhattacharya, Natesh S. Pillai, and David Dunson. 2014. “Posterior Contraction in Sparse Bayesian Factor Models for Massive Covariance Matrices.” The Annals of Statistics 42 (3): 1102–30.
Shalizi, Cosma Rohilla. 2009. “Dynamics of Bayesian Updating with Dependent Data and Misspecified Models.” Electronic Journal of Statistics 3: 1039–74.
Vansteelandt, Stijn, Maarten Bekaert, and Gerda Claeskens. 2012. “On Model Selection and Model Misspecification in Causal Inference.” Statistical Methods in Medical Research 21 (1): 7–30.
Walker, Stephen G. 2013. “Bayesian Inference with Misspecified Models.” Journal of Statistical Planning and Inference 143 (10): 1621–33.
Wang, Yixin, and David Blei. 2019. “Variational Bayes Under Model Misspecification.” In Advances in Neural Information Processing Systems. Vol. 32. Curran Associates, Inc.