Undirected graphical models

September 19, 2017 — October 27, 2019

algebra
graphical models
machine learning
networks
probability
statistics

a.k.a Markov random fields, Markov random networks. (other types?)

I would like to know about spatial Poisson random fields, Markov random fields, Bernoulli random fields, especially for discrete multivariate sequences. Gibbs and Boltzmann distribution inference.

Wasserman’s explanation of the use case is good: Estimating Undirected Graphs Under Weak Assumptions

1 References

Altun, Smola, and Hofmann. 2004. “Exponential Families for Conditional Random Fields.” In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. UAI ’04.
Baddeley, Adrian, and Møller. 1989. “Nearest-Neighbour Markov Point Processes and Random Sets.” International Statistical Review / Revue Internationale de Statistique.
Baddeley, Adrian J, Møller, and Waagepetersen. 2000. “Non- and Semi-Parametric Estimation of Interaction in Inhomogeneous Point Patterns.” Statistica Neerlandica.
Baddeley, A. J., and Van Lieshout. 1995. “Area-Interaction Point Processes.” Annals of the Institute of Statistical Mathematics.
Baddeley, A. J., Van Lieshout, and Møller. 1996. “Markov Properties of Cluster Processes.” Advances in Applied Probability.
Bartolucci, and Besag. 2002. “A Recursive Algorithm for Markov Random Fields.” Biometrika.
Besag. 1974. “Spatial Interaction and the Statistical Analysis of Lattice Systems.” Journal of the Royal Statistical Society. Series B (Methodological).
———. 1975. “Statistical Analysis of Non-Lattice Data.” Journal of the Royal Statistical Society. Series D (The Statistician).
———. 1986. “On the Statistical Analysis of Dirty Pictures.” Journal of the Royal Statistical Society. Series B (Methodological).
Blake, Kohli, and Rother, eds. 2011. Markov Random Fields for Vision and Image Processing.
Boyd. 2010. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers.
Bu, and Lederer. 2017. “Integrating Additional Knowledge Into Estimation of Graphical Models.” arXiv:1704.02739 [Stat].
Celeux, Forbes, and Peyrard. 2003. “EM Procedures Using Mean Field-Like Approximations for Markov Model-Based Image Segmentation.” Pattern Recognition.
Cevher, Duarte, Hegde, et al. 2009. “Sparse Signal Recovery Using Markov Random Fields.” In Advances in Neural Information Processing Systems.
Clifford. 1990. “Markov random fields in statistics.” In Disorder in Physical Systems: A Volume in Honour of John Hammersley.
Crisan, and Míguez. 2014. “Particle-Kernel Estimation of the Filter Density in State-Space Models.” Bernoulli.
Fallat, Lauritzen, Sadeghi, et al. 2017. “Total Positivity in Markov Structures.” The Annals of Statistics.
Forbes, and Peyrard. 2003. “Hidden Markov Random Field Model Selection Criteria Based on Mean Field-Like Approximations.” IEEE Transactions on Pattern Analysis and Machine Intelligence.
Fridman. 2003. “Mixed Markov Models.” Proceedings of the National Academy of Sciences.
Friedman, Hastie, and Tibshirani. 2008. “Sparse Inverse Covariance Estimation with the Graphical Lasso.” Biostatistics.
Friel, and Rue. 2007. “Recursive Computing and Simulation-Free Inference for General Factorizable Models.” Biometrika.
Geyer. 1991. “Markov Chain Monte Carlo Maximum Likelihood.”
Geyer, and Møller. 1994. “Simulation Procedures and Likelihood Inference for Spatial Point Processes.” Scandinavian Journal of Statistics.
Gogate, Webb, and Domingos. 2010. “Learning Efficient Markov Networks.” In Advances in Neural Information Processing Systems.
Goldberg. 2013. “Higher Order Markov Random Fields for Independent Sets.” arXiv:1301.1762 [Math-Ph].
Grenander. 1989. “Advances in Pattern Theory.” The Annals of Statistics.
Griffeath. 1976. “Introduction to Random Fields.” In Denumerable Markov Chains. Graduate Texts in Mathematics 40.
Häggström, van Lieshout, and Møller. 1999. “Characterization Results and Markov Chain Monte Carlo Algorithms Including Exact Simulation for Some Spatial Point Processes.” Bernoulli.
Heckerman, Chickering, Meek, et al. 2000. “Dependency Networks for Inference, Collaborative Filtering, and Data Visualization.” Journal of Machine Learning Research.
Hinton, Osindero, and Bao. 2005. “Learning Causally Linked Markov Random Fields.” In Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics.
Jensen, and Møller. 1991. “Pseudolikelihood for Exponential Family Models of Spatial Point Processes.” The Annals of Applied Probability.
Jordan, Michael Irwin. 1999. Learning in Graphical Models.
Jordan, Michael I. 2004. “Graphical Models.” Statistical Science.
Jordan, Michael I., Ghahramani, Jaakkola, et al. 1999. “An Introduction to Variational Methods for Graphical Models.” Machine Learning.
Jordan, Michael I., and Weiss. 2002a. “Graphical Models: Probabilistic Inference.” The Handbook of Brain Theory and Neural Networks.
———. 2002b. “Probabilistic Inference in Graphical Models.” Handbook of Neural Networks and Brain Theory.
Kamenetsky. 2010. “Ising Graphical Model.”
Karlin, and Rinott. 1980. “Classes of Orderings of Measures and Related Correlation Inequalities. I. Multivariate Totally Positive Distributions.” Journal of Multivariate Analysis.
Khoshgnauz. 2012. “Learning Markov Network Structure Using Brownian Distance Covariance.” arXiv:1206.6361 [Cs, Stat].
Kindermann, Ross, and Snell. 1980. Markov Random Fields and Their Applications. Contemporary Mathematics.
Kindermann, Ross P., and Snell. 1980. “On the Relation Between Markov Random Fields and Social Networks.” The Journal of Mathematical Sociology.
Krämer, Schäfer, and Boulesteix. 2009. “Regularized Estimation of Large-Scale Gene Association Networks Using Graphical Gaussian Models.” BMC Bioinformatics.
Krause, and Guestrin. 2009. “Optimal Value of Information in Graphical Models.” J. Artif. Int. Res.
Kschischang, Frey, and Loeliger. 2001. “Factor Graphs and the Sum-Product Algorithm.” IEEE Transactions on Information Theory.
Lauritzen, Steffen L. 1996. Graphical Models. Oxford Statistical Science Series.
Lauritzen, S. L., and Spiegelhalter. 1988. “Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems.” Journal of the Royal Statistical Society. Series B (Methodological).
Lavrenko, and Pickens. 2003a. “Music Modeling with Random Fields.” In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval.
———. 2003b. “Polyphonic Music Modeling with Random Fields.” In Proceedings of the Eleventh ACM International Conference on Multimedia.
Lederer. 2016. “Graphical Models for Discrete and Continuous Data.” arXiv:1609.05551 [Math, Stat].
Lee, Ganapathi, and Koller. 2006. “Efficient Structure Learning of Markov Networks Using $ L_1 $-Regularization.” In Advances in Neural Information Processing Systems.
Liu, Han, Yuan, et al. 2012a. “The Nonparanormal SKEPTIC.” arXiv:1206.6488 [Cs, Stat].
———, et al. 2012b. “High-Dimensional Semiparametric Gaussian Copula Graphical Models.” The Annals of Statistics.
Liu, Lafferty, and Wasserman. 2009. “The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs.” Journal of Machine Learning Research.
Liu, Roeder, and Wasserman. 2010. “Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models.” In Advances in Neural Information Processing Systems 23.
Loeliger. 2004. “An Introduction to Factor Graphs.” IEEE Signal Processing Magazine.
Maddage, Li, and Kankanhalli. 2006. “Music Structure Based Vector Space Retrieval.” In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.
Malioutov, Johnson, and Willsky. 2006. “Walk-Sums and Belief Propagation in Gaussian Graphical Models.” Journal of Machine Learning Research.
Mazumder, and Hastie. 2012. “The graphical lasso: New insights and alternatives.” Electronic Journal of Statistics.
McCallum. 2012. “Efficiently Inducing Features of Conditional Random Fields.” arXiv:1212.2504 [Cs, Stat].
Meinshausen, and Bühlmann. 2006. “High-Dimensional Graphs and Variable Selection with the Lasso.” The Annals of Statistics.
Mihalkova, and Mooney. 2007. “Bottom-up Learning of Markov Logic Network Structure.” In Proceedings of the 24th International Conference on Machine Learning.
Montanari. 2011. “Lecture Notes for Stat 375 Inference in Graphical Models.”
Morgan, Barjasteh, Lampe, et al. 2014. “The Entropy of Attention and Popularity in Youtube Videos.” arXiv:1412.1185 [Physics].
Murphy. 2012a. Machine learning: a probabilistic perspective. Adaptive computation and machine learning series.
———. 2012b. “Undirected graphical models (Markov random fields).” In Machine Learning: A Probabilistic Perspective.
Osokin, Vetrov, and Kolmogorov. 2011. “Submodular Decomposition Framework for Inference in Associative Markov Networks with Global Constraints.” In 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Pickens, and Iliopoulos. 2005. “Markov Random Fields and Maximum Entropy Modeling for Music Information Retrieval.” In ISMIR.
Pollard. 2004. “Hammersley-Clifford Theorem for Markov Random Fields.”
Ranzato. 2013. “Modeling Natural Images Using Gated MRFs.” IEEE Transactions on Pattern Analysis and Machine Intelligence.
Ravikumar, Wainwright, and Lafferty. 2010. “High-Dimensional Ising Model Selection Using ℓ1-Regularized Logistic Regression.” The Annals of Statistics.
Reeves, and Pettitt. 2004. “Efficient Recursions for General Factorisable Models.” Biometrika.
Richardson, and Domingos. 2006. “Markov Logic Networks.” Machine Learning.
Ripley, and Kelly. 1977. “Markov Point Processes.” Journal of the London Mathematical Society.
Schmidt, and Murphy. 2010. “Convex Structure Learning in Log-Linear Models: Beyond Pairwise Potentials.” In International Conference on Artificial Intelligence and Statistics.
Slawski, and Hein. 2015. “Estimation of Positive Definite M-Matrices and Structure Learning for Attractive Gaussian Markov Random Fields.” Linear Algebra and Its Applications, Special issue on Statistics,.
Studený. 1997. “A Recovery Algorithm for Chain Graphs.” International Journal of Approximate Reasoning, Uncertainty in AI (UAI’96) Conference,.
———. 2005. Probabilistic Conditional Independence Structures. Information Science and Statistics.
Sutton, and McCallum. 2010. “An Introduction to Conditional Random Fields.” arXiv:1011.4088.
Tansey, Padilla, Suggala, et al. 2015. “Vector-Space Markov Random Fields via Exponential Families.” In Journal of Machine Learning Research.
Vetrov, and Osokin. 2011. “Graph Preserving Label Decomposition in Discrete MRFs with Selfish Potentials.” In NIPS Workshop on Discrete Optimization in Machine Learning (DISCML NIPS).
Visweswaran, and Cooper. 2014. “Counting Markov Blanket Structures.” arXiv:1407.2483 [Cs, Stat].
Wainwright, Martin, and Jordan. 2005. “A Variational Principle for Graphical Models.” In New Directions in Statistical Signal Processing.
Wainwright, Martin J., and Jordan. 2008. Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning.
Wang, Komodakis, and Paragios. 2013. “Markov Random Field Modeling, Inference & Learning in Computer Vision & Image Understanding: A Survey.” Computer Vision and Image Understanding.
Wasserman, Kolar, and Rinaldo. 2013. “Estimating Undirected Graphs Under Weak Assumptions.” arXiv:1309.6933 [Cs, Math, Stat].
Wu, Srikant, and Ni. 2012. “Learning Graph Structures in Discrete Markov Random Fields.” In INFOCOM Workshops.
———. 2013. “Learning Loosely Connected Markov Random Fields.” Stochastic Systems.
Yedidia, Freeman, and Weiss. 2003. “Understanding Belief Propagation and Its Generalizations.” In Exploring Artificial Intelligence in the New Millennium.
———. 2005. “Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms.” IEEE Transactions on Information Theory.
Zhao, Liu, Roeder, et al. 2012. “The Huge Package for High-Dimensional Undirected Graph Estimation in R.” Journal of Machine Learning Research : JMLR.
Zhu, and Fan. 2022. “A Synthetic Likelihood Approach for Intractable Markov Random Fields.” Computational Statistics.