Energy based models

Inference with kinda-tractable un-normalized densities



I don’t actually know what fits under this heading, but it sounds like it is simply inference for undirected graphical models? Or is there something distinct going on here?

Descending the local energy gradient to a more probable configuration

References

Che, Tong, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, and Yoshua Bengio. 2020. “Your GAN Is Secretly an Energy-Based Model and You Should Use Discriminator Driven Latent Sampling.” arXiv:2003.06060 [cs, Stat], March. http://arxiv.org/abs/2003.06060.
Clifford, P. 1990. “Markov random fields in statistics.” In Disorder in Physical Systems: A Volume in Honour of John Hammersley, edited by G. R. Grimmett and D. J. A. Welsh. Oxford England : New York: Oxford University Press.
Hinton, Geoffrey. 2010. “A Practical Guide to Training Restricted Boltzmann Machines.” In Neural Networks: Tricks of the Trade, 9:926. Lecture Notes in Computer Science 7700. Springer Berlin Heidelberg. http://www.csri.utoronto.ca/~hinton/absps/guideTR.pdf.
LeCun, Yann, Sumit Chopra, Raia Hadsell, M. Ranzato, and F. Huang. 2006. “A Tutorial on Energy-Based Learning.” In Predicting Structured Data. http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf.
Montavon, Grégoire, Klaus-Robert Müller, and Marco Cuturi. 2016. “Wasserstein Training of Restricted Boltzmann Machines.” In Advances in Neural Information Processing Systems 29, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 3711–19. Curran Associates, Inc. http://papers.nips.cc/paper/6248-wasserstein-training-of-restricted-boltzmann-machines.pdf.
Pollard, Dave. 2004. “Hammersley-Clifford Theorem for Markov Random Fields.”
Salakhutdinov, Ruslan. 2015. “Learning Deep Generative Models.” Annual Review of Statistics and Its Application 2 (1): 361–85. https://doi.org/10.1146/annurev-statistics-010814-020120.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.