M-open, M-complete, M-closed

May 30, 2016 — July 23, 2023

Suspiciously similar content

Placeholder.

I came across the M-open concept and found it useful to mention, though I haven’t had time to really delve into it. M-open, M-complete, and M-closed describe different relations between our hypothesis class and reality. Basically, do we have the true model in our hypothesis class? (Spoiler: no, we usually don’t, except with synthetic data.) If not, what does our estimation procedure provide us?

Fancy folks write this as $M$ -open etc., but life is too short for indulgent typography.

Le and Clarke (2017) summarises:

For the sake of completeness, we recall that Bernardo and Smith (Bernardo and Smith 2000) define M-closed problems as those for which a true model can be identified and written down but is one amongst finitely many models from which an analyst has to choose. By contrast, M-complete problems are those in which a true model (sometimes called a belief model) exists but is inaccessible in the sense that even though it can be conceptualised it cannot be written down or at least cannot be used directly. Effectively this means that other surrogate models must be identified and used for inferential purposes. M-open problems according to Bernardo and Smith (2000) are those problems where a true model exists but cannot be specified at all.

They also mention Clyde and Iversen (2013) as a useful resource.

You’ll notice that many references find this concept interesting because of its application to Bayesian model stacking.

Related: likelihood principle, decision-theory, black swans, misspecified models, Robust bayes …

1 Gibbs posteriors

Gibbs posteriors seem to be an attempt to address the M-open problem, by removing the need for a valid likelihood.

2 References

Berger, Wolpert, Bayarri, et al. 1988. “The Likelihood Principle.” Lecture Notes-Monograph Series.

Bernardo, and Smith. 2000. Bayesian Theory.

Clarke. 2003. “Comparing Bayes Model Averaging and Stacking When Model Approximation Error Cannot Be Ignored.” The Journal of Machine Learning Research.

Clyde, and Iversen. 2013. “Bayesian Model Averaging in the M-Open Framework.” In Bayesian Theory and Applications.

Dellaporta, Knoblauch, Damoulas, et al. 2022. “Robust Bayesian Inference for Simulator-Based Models via the MMD Posterior Bootstrap.” arXiv:2202.04744 [Cs, Stat].

Jansen. n.d. “Robust Bayesian Inference Under Model Misspeciﬁcation.”

Knoblauch, Jewson, and Damoulas. 2019. “Generalized Variational Inference: Three Arguments for Deriving New Posteriors.”

———. 2022. “An Optimization-Centric View on Bayes’ Rule: Reviewing and Generalizing Variational Inference.” Journal of Machine Learning Research.

Le, and Clarke. 2017. “A Bayes Interpretation of Stacking for M-Complete and M-Open Settings.” Bayesian Analysis.

Lyddon, Walker, and Holmes. 2018. “Nonparametric Learning from Bayesian Models with Randomized Objective Functions.” In Proceedings of the 32nd International Conference on Neural Information Processing Systems. NIPS’18.

Masegosa. 2020. “Learning Under Model Misspecification: Applications to Variational and Ensemble Methods.” In Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS’20.

Matsubara, Knoblauch, Briol, et al. 2022. “Robust Generalised Bayesian Inference for Intractable Likelihoods.” Journal of the Royal Statistical Society Series B: Statistical Methodology.

Minka. 2002. “Bayesian Model Averaging Is Not Model Combination.”

Pacchiardi, and Dutta. 2022. “Generalized Bayesian Likelihood-Free Inference Using Scoring Rules Estimators.” arXiv:2104.03889 [Stat].

Schmon, Cannon, and Knoblauch. 2021. “Generalized Posteriors in Approximate Bayesian Computation.” arXiv:2011.08644 [Stat].

Shirvaikar, Walker, and Holmes. 2024. “A General Framework for Probabilistic Model Uncertainty.”

Yao, Vehtari, Simpson, et al. 2018. “Using Stacking to Average Bayesian Predictive Distributions.” Bayesian Analysis.