Simulation based inference

a.k.a. Approximate Bayesian Computation or Likelihood-Free-Inference


Simulation-based inference, likelihood-free inference, and approximate Bayesian Computation are all terrible descriptions. There are many ways that inference can be based upon simulations, many types of freedom from likelihood and many ways to approximate Bayesian computation. However, all these terms together refer to a particular thing.

TBD: relationship between this and indirect inference. They look very similar but tend not to cite each other. Is this a technical or sociological difference?

Miles Cranmer’s Introduction to Simulation-based inference.

Baydin, Atılım Güneş, Lei Shao, Wahid Bhimji, Lukas Heinrich, Lawrence Meadows, Jialin Liu, Andreas Munk, et al. 2019. “Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale.” In. http://arxiv.org/abs/1907.03382.

Cranmer, Kyle, Johann Brehmer, and Gilles Louppe. 2020. “The Frontier of Simulation-Based Inference.” Proceedings of the National Academy of Sciences, May. https://doi.org/10.1073/pnas.1912789117.

Cranmer, Kyle, Juan Pavez, and Gilles Louppe. 2015. “Approximating Likelihood Ratios with Calibrated Discriminative Classifiers,” June. https://arxiv.org/abs/1506.02169v2.

Diggle, Peter J., and Richard J. Gratton. 1984. “Monte Carlo Methods of Inference for Implicit Statistical Models.” Journal of the Royal Statistical Society: Series B (Methodological) 46 (2): 193–212. https://doi.org/10.1111/j.2517-6161.1984.tb01290.x.

Durkan, Conor, George Papamakarios, and Iain Murray. 2018. “Sequential Neural Methods for Likelihood-Free Inference,” 9. https://arxiv.org/abs/1811.08723v1.

Fan, Yanan, David J. Nott, and Scott A. Sisson. 2013. “Approximate Bayesian Computation via Regression Density Estimation.” Stat 2 (1): 34–48. https://doi.org/10.1002/sta4.15.

Forneron, Jean-Jacques, and Serena Ng. 2015. “The ABC of Simulation Estimation with Auxiliary Statistics,” January. http://arxiv.org/abs/1501.01265.

Gourieroux, Christian, and Alain Monfort. 1993. “Simulation-Based Inference: A Survey with Special Reference to Panel Data Models.” Journal of Econometrics 59 (1–2): 5–33. https://doi.org/10.1016/0304-4076(93)90037-6.

Izbicki, Rafael, Ann B. Lee, and Taylor Pospisil. 2019. “ABC–CDE: Toward Approximate Bayesian Computation with Complex High-Dimensional Data and Limited Simulations.” Journal of Computational and Graphical Statistics 28 (3): 481–92. https://doi.org/10.1080/10618600.2018.1546594.

Le, Tuan Anh, Atılım Güneş Baydin, and Frank Wood. 2017. “Inference Compilation and Universal Probabilistic Programming.” In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 54:1338–48. Proceedings of Machine Learning Research. Fort Lauderdale, FL, USA: PMLR. http://arxiv.org/abs/1610.09900.

Lueckmann, Jan-Matthis, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H. Macke. 2019. “Likelihood-Free Inference with Emulator Networks.” In Symposium on Advances in Approximate Bayesian Inference, 32–53. http://proceedings.mlr.press/v96/lueckmann19a.html.

Meeds, Edward, and Max Welling. 2014. “GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation.” In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, 593–602. UAI’14. Arlington, Virginia, USA: AUAI Press. https://arxiv.org/abs/1401.2838v1.

Mohamed, Shakir, and Balaji Lakshminarayanan. 2016. “Learning in Implicit Generative Models,” November. https://arxiv.org/abs/1610.03483v4.

Neal, Radford. 2008. “Computing Likelihood Functions for High-Energy Physics Experiments When Distributions Are Defined by Simulators with Nuisance Parameters.” https://doi.org/10.5170/CERN-2008-001.111.

Papamakarios, George, and Iain Murray. 2016. “Fast \Epsilon -Free Inference of Simulation Models with Bayesian Conditional Density Estimation.” In Advances in Neural Information Processing Systems 29, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 1028–36. Curran Associates, Inc. http://papers.nips.cc/paper/6084-fast-free-inference-of-simulation-models-with-bayesian-conditional-density-estimation.pdf.

Rubin, Donald B. 1984. “Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician.” Annals of Statistics 12 (4): 1151–72. https://doi.org/10.1214/aos/1176346785.

Sisson, S. A., Y. Fan, and Mark M. Tanaka. 2007. “Sequential Monte Carlo Without Likelihoods.” Proceedings of the National Academy of Sciences 104 (6): 1760–5. https://doi.org/10.1073/pnas.0607208104.

Sisson, Scott A., Yanan Fan, and Mark Beaumont. 2018. Handbook of Approximate Bayesian Computation. CRC Press. http://books.google.com?id=9QhpDwAAQBAJ.

Stoye, Markus, Johann Brehmer, Gilles Louppe, Juan Pavez, and Kyle Cranmer. 2018. “Likelihood-Free Inference with an Improved Cross-Entropy Estimator,” August. http://arxiv.org/abs/1808.00973.

Tran, Dustin, Rajesh Ranganath, and David Blei. 2017. “Hierarchical Implicit Models and Likelihood-Free Variational Inference.” In Advances in Neural Information Processing Systems 30, edited by I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, 5523–33. Curran Associates, Inc. http://papers.nips.cc/paper/7136-hierarchical-implicit-models-and-likelihood-free-variational-inference.pdf.