Approximate Bayesian Computation

Posterior updates without likelihood



Approximate Bayesian Computation is a terribly underspecified description. There are many ways that inference can be based upon simulations, many types of freedom from likelihood and many ways to approximate Bayesian computation. This page is about the dominant use of that term, which is the use of Simulation-based inference to do Bayes updates where the likelihood is not available but where we can simulate from the generative model.

Obviously there are other ways you can approximate Bayesian computation — see e.g. variational Bayes.

TBD: relationship between this and simulation-based inference in a frequentist setting, often called indirect inference. They look similar but tend not to cite each other. Is this a technical or sociological hurdle?

Miles Cranmer’s Introduction to Simulation-based inference.

SMC for ABC

One can solve for ABC using Sequential Monte Carlo. TBD.

Bayesian Synthetic Likelihood

TBD. Something about assuming the summary statistic is close to jointly Gaussian.

References

Baydin, Atılım Güneş, Lei Shao, Wahid Bhimji, Lukas Heinrich, Lawrence Meadows, Jialin Liu, Andreas Munk, et al. 2019. “Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale.” In. http://arxiv.org/abs/1907.03382.
Cranmer, Kyle, Johann Brehmer, and Gilles Louppe. 2020. “The Frontier of Simulation-Based Inference.” Proceedings of the National Academy of Sciences, May. https://doi.org/10.1073/pnas.1912789117.
Cranmer, Kyle, Juan Pavez, and Gilles Louppe. 2015. “Approximating Likelihood Ratios with Calibrated Discriminative Classifiers,” June. https://arxiv.org/abs/1506.02169v2.
Diggle, Peter J., and Richard J. Gratton. 1984. “Monte Carlo Methods of Inference for Implicit Statistical Models.” Journal of the Royal Statistical Society: Series B (Methodological) 46 (2): 193–212. https://doi.org/10.1111/j.2517-6161.1984.tb01290.x.
Drovandi, Christopher C., Clara Grazian, Kerrie Mengersen, and Christian Robert. 2018. “Approximating the Likelihood in Approximate Bayesian Computation.” March 18, 2018. http://arxiv.org/abs/1803.06645.
Drovandi, Christopher, and David T. Frazier. 2021. “A Comparison of Likelihood-Free Methods With and Without Summary Statistics.” March 3, 2021. http://arxiv.org/abs/2103.02407.
Durkan, Conor, George Papamakarios, and Iain Murray. 2018. “Sequential Neural Methods for Likelihood-Free Inference,” 9. https://arxiv.org/abs/1811.08723v1.
Fan, Yanan, David J. Nott, and Scott A. Sisson. 2013. “Approximate Bayesian Computation via Regression Density Estimation.” Stat 2 (1): 34–48. https://doi.org/10.1002/sta4.15.
Forneron, Jean-Jacques, and Serena Ng. 2015. “The ABC of Simulation Estimation with Auxiliary Statistics.” January 6, 2015. http://arxiv.org/abs/1501.01265.
Frazier, David T., and Christopher Drovandi. 2021. “Robust Approximate Bayesian Inference With Synthetic Likelihood.” Journal of Computational and Graphical Statistics 0 (0): 1–19. https://doi.org/10.1080/10618600.2021.1875839.
Frazier, David T., David J. Nott, Christopher Drovandi, and Robert Kohn. 2021. “Bayesian Inference Using Synthetic Likelihood: Asymptotics and Adjustments.” March 12, 2021. http://arxiv.org/abs/1902.04827.
Gourieroux, Christian, and Alain Monfort. 1993. “Simulation-Based Inference: A Survey with Special Reference to Panel Data Models.” Journal of Econometrics 59 (1–2): 5–33. https://doi.org/10.1016/0304-4076(93)90037-6.
Izbicki, Rafael, Ann B. Lee, and Taylor Pospisil. 2019. ABCCDE: Toward Approximate Bayesian Computation With Complex High-Dimensional Data and Limited Simulations.” Journal of Computational and Graphical Statistics 28 (3): 481–92. https://doi.org/10.1080/10618600.2018.1546594.
Le, Tuan Anh, Atılım Güneş Baydin, and Frank Wood. 2017. “Inference Compilation and Universal Probabilistic Programming.” In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 54:1338–48. Proceedings of Machine Learning Research. Fort Lauderdale, FL, USA: PMLR. http://arxiv.org/abs/1610.09900.
Lueckmann, Jan-Matthis, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H. Macke. 2019. “Likelihood-Free Inference with Emulator Networks.” In Symposium on Advances in Approximate Bayesian Inference, 32–53. http://proceedings.mlr.press/v96/lueckmann19a.html.
Meeds, Edward, and Max Welling. 2014. GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation.” In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, 593–602. UAI’14. Arlington, Virginia, USA: AUAI Press. https://arxiv.org/abs/1401.2838v1.
Mohamed, Shakir, and Balaji Lakshminarayanan. 2016. “Learning in Implicit Generative Models,” November. https://arxiv.org/abs/1610.03483v4.
Neal, Radford. 2008. “Computing Likelihood Functions for High-Energy Physics Experiments When Distributions Are Defined by Simulators with Nuisance Parameters.” https://doi.org/10.5170/CERN-2008-001.111.
Papamakarios, George, and Iain Murray. 2016. “Fast ε-Free Inference of Simulation Models with Bayesian Conditional Density Estimation.” In Advances in Neural Information Processing Systems 29, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 1028–36. Curran Associates, Inc. http://papers.nips.cc/paper/6084-fast-free-inference-of-simulation-models-with-bayesian-conditional-density-estimation.pdf.
Rubin, Donald B. 1984. “Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician.” Annals of Statistics 12 (4): 1151–72. https://doi.org/10.1214/aos/1176346785.
Sisson, S. A., Y. Fan, and Mark M. Tanaka. 2007. “Sequential Monte Carlo Without Likelihoods.” Proceedings of the National Academy of Sciences 104 (6): 1760–65. https://doi.org/10.1073/pnas.0607208104.
Sisson, Scott A., Yanan Fan, and Mark Beaumont. 2018. Handbook of Approximate Bayesian Computation. CRC Press. http://books.google.com?id=9QhpDwAAQBAJ.
Stoye, Markus, Johann Brehmer, Gilles Louppe, Juan Pavez, and Kyle Cranmer. 2018. “Likelihood-Free Inference with an Improved Cross-Entropy Estimator.” August 2, 2018. http://arxiv.org/abs/1808.00973.
Tran, Dustin, Rajesh Ranganath, and David Blei. 2017. “Hierarchical Implicit Models and Likelihood-Free Variational Inference.” In Advances in Neural Information Processing Systems 30, edited by I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, 5523–33. Curran Associates, Inc. http://papers.nips.cc/paper/7136-hierarchical-implicit-models-and-likelihood-free-variational-inference.pdf.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.