Likelihood free inference

A term which seems to have a couple of distinct uses.

Here, I mostly mean this in the sense of trying to approximate intractable likelihoods, possibly by Monte Carlo simulations from a generative model. There seems also to be a school which would like to use this term for methods which make no reference to probability densities whatever. See this grab bag.

Within my use of the term, there is much to find out. Is it mostly modern reframing and rebranding of indirect inference? Is there something more? Techniques like GANS and reparameterisation methods would fit in that classification.

The archetypal method is presumably Approximate Bayesian Computation, which could make a case for ownership the recent history of the term in the form of simulation based inference. Maybe check the reading list for ABC toolkit ELFI.

Cranmer, Brehmer, and Louppe (2020) attempt to develop a taxonomy. They certainly make likelihood-free methods sound popular in machine learning for physics.

There is another sense of “likelihood free’, where you try to do inference which targets something other than the classic KL divergence as a loss function. This also looks interesting, and essentially orthogonal to the question of approximating the likelihood, e.g. Matsubara et al. (2021). A method that works to approximate an intractable likelihood function might also work to approximate some other intractable loss. If you do lots of predictive point estimation, this second use of the term seems like a waste of a term. In that area, loss functions are pluggable anyway. However, in the case of Bayesian inference, or more generally, distributional inference, complications ensue.