# Likelihood free inference

Finding the target without directly inspecting the likelihood of the current guess

A term which seems to have a couple of distinct uses.

Here, I mostly mean this in the sense of trying to approximate intractable likelihoods, possibly by Monte Carlo simulations from a generative model. There seems also to be a school which would like to use this term for methods which make no reference to probability densities whatever. See this grab bag.

Within my use of the term, there is much to find out. Is it mostly modern reframing and rebranding of indirect inference? Is there something more? Techniques like GANS and reparameterisation methods would fit in that classification.

The archetypal method is presumably Approximate Bayesian Computation, which could make a case for ownership the recent history of the term in the form of simulation based inference. Maybe check the reading list for ABC toolkit ELFI.

Cranmer, Brehmer, and Louppe (2020) attempt to develop a taxonomy. They certainly make likelihood-free methods sound popular in machine learning for physics.

Cranmer et al’s taxonomy of “simulation based” approaches.

There is another sense of “likelihood free’, where you try to do inference which targets something other than the classic KL divergence as a loss function. This also looks interesting, and essentially orthogonal to the question of approximating the likelihood, e.g. Matsubara et al. (2021). A method that works to approximate an intractable likelihood function might also work to approximate some other intractable loss. If you do lots of predictive point estimation, this second use of the term seems like a waste of a term. In that area, loss functions are pluggable anyway. However, in the case of Bayesian inference, or more generally, distributional inference, complications ensue.

## By density ratio estimation

See density ratio.

## References

Babtie, Ann C., Paul Kirk, and Michael P. H. Stumpf. 2014. Proceedings of the National Academy of Sciences 111 (52): 18507–12.
Batz, Philipp, Andreas Ruttor, and Manfred Opper. 2017. arXiv:1702.05390 [Physics, Stat], February.
Brehmer, Johann, Gilles Louppe, Juan Pavez, and Kyle Cranmer. 2020. Proceedings of the National Academy of Sciences 117 (10): 5242–49.
Bretó, Carles, Daihai He, Edward L. Ionides, and Aaron A. King. 2009. The Annals of Applied Statistics 3 (1): 319–48.
Castro, Pablo de, and Tommaso Dorigo. 2019. Computer Physics Communications 244 (November): 170–79.
Cauchemez, Simon, and Neil M. Ferguson. 2008. Journal of The Royal Society Interface 5 (25): 885–97.
Clark, James S., and Ottar N. Bjørnstad. 2004. Ecology 85 (11): 3140–50.
Commandeur, Jacques J. F., Siem Jan Koopman, and Marius Ooms. 2011. Journal of Statistical Software 41 (1).
Cook, Alex R., Wilfred Otten, Glenn Marion, Gavin J. Gibson, and Christopher A. Gilligan. 2007. Proceedings of the National Academy of Sciences 104 (51): 20392–97.
Cox, D. R., and Christiana Kartsonaki. 2012. Biometrika 99 (3): 741–47.
Cranmer, Kyle, Johann Brehmer, and Gilles Louppe. 2020. Proceedings of the National Academy of Sciences, May.
Creel, Michael, and Dennis Kristensen. 2012. The Econometrics Journal 15 (3): 490–515.
———. 2013. UFAE and IAE Working Paper 931.13. Unitat de Fonaments de l’Anàlisi Econòmica (UAB) and Institut d’Anàlisi Econòmica (CSIC).
Czellar, Veronika, and Elvezio Ronchetti. 2010. Biometrika 97 (3): 621–30.
Dellaporta, Charita, Jeremias Knoblauch, Theodoros Damoulas, and François-Xavier Briol. 2022. arXiv:2202.04744 [Cs, Stat], February.
Didelot, Xavier, Richard G. Everitt, Adam M. Johansen, and Daniel J. Lawson. 2011. Bayesian Analysis 6 (1): 49–76.
Dridi, Ramdan, Alain Guay, and Eric Renault. 2007. Journal of Econometrics, The interface between econometrics and economic theory, 136 (2): 397–430.
Drovandi, Christopher, and David T. Frazier. 2021. arXiv:2103.02407 [Stat], March.
Efron, Bradley. 2010. Statistical Science 25 (2): 145–57.
Fong, Edwin, Simon Lyddon, and Chris Holmes. 2019. arXiv:1902.03175 [Cs, Stat], August.
Forneron, Jean-Jacques, and Serena Ng. 2015. arXiv:1501.01265 [Stat], January.
Gallant, A. Ronald, David Hsieh, and George Tauchen. 1997. Journal of Econometrics 81 (1): 159–92.
Gallant, A. Ronald, and George Tauchen. 1996. Econometric Theory 12 (04): 657–81.
Genton, Marc G, and Elvezio Ronchetti. 2003. Journal of the American Statistical Association 98 (461): 67–76.
Gourieroux, Christian, and Alain Monfort. 1993. Journal of Econometrics 59 (1–2): 5–33.
Gourieroux, C., A. Monfort, and E. Renault. 1993. Journal of Applied Econometrics 8 (December): S85–118.
Grelaud, Aude, Christian P. Robert, Jean-Michel Marin, François Rodolphe, and Jean-François Taly. 2009. Bayesian Analysis 4 (2): 317–35.
Gutmann, Michael U., and Jukka Corander. 2016. “Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models.” The Journal of Machine Learning Research 17 (1): 4256–4302.
He, Daihai, Edward L. Ionides, and Aaron A. King. 2010. Journal of The Royal Society Interface 7 (43): 271–83.
Hermans, Joeri, Volodimir Begy, and Gilles Louppe. 2020. arXiv:1903.04057 [Cs, Stat], February.
Ionides, E. L., C. Bretó, and A. A. King. 2006. Proceedings of the National Academy of Sciences 103 (49): 18438–43.
Ionides, Edward L., Anindya Bhadra, Yves Atchadé, and Aaron King. 2011. The Annals of Statistics 39 (3): 1776–1802.
Jiang, Wenxin, and Bruce Turnbull. 2004. Statistical Science 19 (2): 239–63.
Kendall, Bruce E., Stephen P. Ellner, Edward McCauley, Simon N. Wood, Cheryl J. Briggs, William W. Murdoch, and Peter Turchin. 2005. Ecological Monographs 75 (2): 259–76.
Lueckmann, Jan-Matthis, Giacomo Bassetto, Theofanis Karaletsos, and Jakob H. Macke. 2019. In Symposium on Advances in Approximate Bayesian Inference, 32–53.
Lyddon, Simon, Stephen Walker, and Chris Holmes. 2018. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2075–85. NIPS’18. Red Hook, NY, USA: Curran Associates Inc.
Matsubara, Takuo, Jeremias Knoblauch, François-Xavier Briol, and Chris J. Oates. 2021. arXiv:2104.07359 [Math, Stat], April.
Miller, Benjamin Kurt, Alex Cole, and Gilles Louppe. n.d. “Simulation-Efﬁcient Marginal Posterior Estimation with Swyft: Stop Wasting Your Precious Time.” In, 9.
Nickl, Richard, and Benedikt M. Pötscher. 2009. Mathematical Methods of Statistics 19, August, 327–64.
Pacchiardi, Lorenzo, and Ritabrata Dutta. 2022. arXiv:2104.03889 [Stat], March.
Roberts, G. O., and O. Stramer. 2001. Biometrika 88 (3): 603–21.
Schmon, Sebastian M., Patrick W. Cannon, and Jeremias Knoblauch. 2021. arXiv:2011.08644 [Stat], February.
Smith, A A. 2008. In The New Palgrave Dictionary of Economics. Palgrave Macmillan.
Smith, A. A. 1993. Journal of Applied Econometrics 8 (S1): S63–84.
Stoye, Markus, Johann Brehmer, Gilles Louppe, Juan Pavez, and Kyle Cranmer. 2018. arXiv:1808.00973 [Hep-Ph, Physics:physics, Stat], August.
Wood, Simon N. 2010. Nature 466 (7310): 1102–4.

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.