Implicit variational inference
Variational inference without densities
May 10, 2024 — October 21, 2024
approximation
metrics
optimization
probabilistic algorithms
probability
statistics
Variational inference using generative models whose density cannot be evaluated. See Variational Inference using Implicit Models.
Even though it does not evaluate likelihoods, implicit VI still seems to use KL divergence as a loss function.
There seems to be a connection to adversarial learning too.
1 Connection to adversarial training
Related concepts, perhaps? Variational interpretation of adversarial losses:
2 Semi-implicit variational inference
Not sure yet. I should check out Conor Hassan’s implementation.
See (Cheng et al. 2024; Lim and Johansen 2024; Moens et al. 2021; Molchanov et al. 2019; Yu et al. 2024; Yu and Zhang 2023).
3 References
Cheng, Yu, Xie, et al. 2024. “Kernel Semi-Implicit Variational Inference.”
Che, Zhang, Sohl-Dickstein, et al. 2020. “Your GAN Is Secretly an Energy-Based Model and You Should Use Discriminator Driven Latent Sampling.” arXiv:2003.06060 [Cs, Stat].
Huszár. 2017. “Variational Inference Using Implicit Distributions.”
Karaletsos. 2016. “Adversarial Message Passing For Graphical Models.”
Lim, and Johansen. 2024. “Particle Semi-Implicit Variational Inference.”
Moens, Ren, Maraval, et al. 2021. “Efficient Semi-Implicit Variational Inference.”
Mohamed, and Rezende. 2015. “Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning.” In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2. NIPS’15.
Molchanov, Kharitonov, Sobolev, et al. 2019. “Doubly Semi-Implicit Variational Inference.” In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics.
Smith, and Loaiza-Maya. 2023. “Implicit Copula Variational Inference.” Journal of Computational and Graphical Statistics.
Tiao, Bonilla, and Ramos. 2018. “Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference.”
Tran, Ranganath, and Blei. 2017. “Hierarchical Implicit Models and Likelihood-Free Variational Inference.” In Advances in Neural Information Processing Systems 30.
Uppal, Stensbo-Smidt, Boomsma, et al. 2023. “Implicit Variational Inference for High-Dimensional Posteriors.”
Yu, Xie, Zhu, et al. 2024. “Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration.” In Proceedings of the 37th International Conference on Neural Information Processing Systems. NIPS ’23.
Yu, and Zhang. 2023. “Semi-Implicit Variational Inference via Score Matching.”