Implicit variational inference

Variational inference without densities

May 10, 2024

approximation
metrics
optimization
probabilistic algorithms
probability
statistics
Figure 1

Variational inference using generative models whose density cannot be evaluated. See Variational Inference using Implicit Models.

Even though it does not evaluate likelihoods implicit VI still seems to use KL divergence as a loss function.

There seems to be a connection to adversarial learning too.

Figure 2: From David Burt and Andrew Foong: Introduction to Implicit Variational Inference.

1 References

Huszár. 2017. Variational Inference Using Implicit Distributions.”
Karaletsos. 2016. Adversarial Message Passing For Graphical Models.”
Mohamed, and Rezende. 2015. “Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning.” In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2. NIPS’15.
Tran, Ranganath, and Blei. 2017. Hierarchical Implicit Models and Likelihood-Free Variational Inference.” In Advances in Neural Information Processing Systems 30.
Uppal, Stensbo-Smidt, Boomsma, et al. 2023. Implicit Variational Inference for High-Dimensional Posteriors.”