Energy based models

Inference with kinda-tractable un-normalized densities

June 7, 2021 — June 7, 2021

Monte Carlo
probabilistic algorithms

I don’t actually know what fits under this heading, but it sounds like it is simply inference for undirected graphical models? Or is there something distinct going on?

Figure 1: Descending the local energy gradient to a more probable configuration

1 References

Bardes, Ponce, and LeCun. 2022. VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning.”
Che, Zhang, Sohl-Dickstein, et al. 2020. Your GAN Is Secretly an Energy-Based Model and You Should Use Discriminator Driven Latent Sampling.” arXiv:2003.06060 [Cs, Stat].
Clifford. 1990. “Markov random fields in statistics.” In Disorder in Physical Systems: A Volume in Honour of John Hammersley.
Hinton. 2010. A Practical Guide to Training Restricted Boltzmann Machines.” In Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science 7700.
LeCun, Chopra, Hadsell, et al. 2006. A Tutorial on Energy-Based Learning.” In Predicting Structured Data.
Montavon, Müller, and Cuturi. 2016. Wasserstein Training of Restricted Boltzmann Machines.” In Advances in Neural Information Processing Systems 29.
Pollard. 2004. “Hammersley-Clifford Theorem for Markov Random Fields.”
Salakhutdinov. 2015. Learning Deep Generative Models.” Annual Review of Statistics and Its Application.