Exponential families


Exponential families! The secret magic at the heart of traditional statistics.

Exponential families are (sorta) probability distributions that just work, in the sense that and the things you would hope you can do with them, you can.

Noted here so I have somewhere to dump notes, but not something I am going to go into right now.

Most interesting models are not exponential families. There are curved exponential families which generalise exponential families in some way that I have not looked into.

Justin Domke summarises exponential families for graphical model usage.

Altun, Yasemin, Alex J. Smola, and Thomas Hofmann. 2004. “Exponential Families for Conditional Random Fields.” In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, 2–9. UAI ’04. Arlington, Virginia, United States: AUAI Press. http://arxiv.org/abs/1207.4131.

Balkema, A. A., and L. de Haan. 1974. “Residual Life Time at Great Age.” The Annals of Probability 2 (5): 792–804. https://doi.org/10.1214/aop/1176996548.

Brown, Lawrence D. 1986. Fundamentals of Statistical Exponential Families: With Applications in Statistical Decision Theory. Lecture Notes-Monograph Series, v. 9. Hayward, Calif: Institute of Mathematical Statistics.

Brown, Lawrence D., T. Tony Cai, and Harrison H. Zhou. 2010. “Nonparametric Regression in Exponential Families.” The Annals of Statistics 38 (4): 2005–46. https://doi.org/10.1214/09-AOS762.

Charpentier, Arthur, and Emmanuel Flachaire. 2019. “Pareto Models for Risk Management,” December. http://arxiv.org/abs/1912.11736.

Efron, Bradley. 1978. “The Geometry of Exponential Families.” The Annals of Statistics 6 (2): 362–76. https://doi.org/10.1214/aos/1176344130.

Fisher, R. A., and L. H. C. Tippett. 1928. “Limiting Forms of the Frequency Distribution of the Largest or Smallest Member of a Sample.” Mathematical Proceedings of the Cambridge Philosophical Society 24 (2): 180–90. https://doi.org/10.1017/S0305004100015681.

Jensen, Jens Ledet, and Jesper Møller. 1991. “Pseudolikelihood for Exponential Family Models of Spatial Point Processes.” The Annals of Applied Probability 1 (3): 445–61. https://doi.org/10.1214/aoap/1177005877.

Jung, Alexander, Sebastian Schmutzhard, and Franz Hlawatsch. 2012. “The RKHS Approach to Minimum Variance Estimation Revisited: Variance Bounds, Sufficient Statistics, and Exponential Families,” October. http://arxiv.org/abs/1210.6516.

Makarov, Mikhail. 2006. “Extreme Value Theory and High Quantile Convergence.” The Journal of Operational Risk 1 (2): 51–57. https://doi.org/10.21314/JOP.2006.009.

McNeil, Alexander J. 1997. “Estimating the Tails of Loss Severity Distributions Using Extreme Value Theory.” ASTIN Bulletin: The Journal of the IAA 27 (1): 117–37. https://doi.org/10.2143/AST.27.1.563210.

Mueller, Ulrich K. 2018. “Refining the Central Limit Theorem Approximation via Extreme Value Theory,” February. http://arxiv.org/abs/1802.00762.

Pickands III, James. 1975. “Statistical Inference Using Extreme Order Statistics.” The Annals of Statistics 3 (1): 119–31. https://doi.org/10.1214/aos/1176343003.

Shen, Xiaotong, Hsin-Cheng Huang, and Jimmy Ye. 2004. “Adaptive Model Selection and Assessment for Exponential Family Distributions.” Technometrics 46 (3): 306–17. https://doi.org/10.1198/004017004000000338.

Tansey, Wesley, Oscar Hernan Madrid Padilla, Arun Sai Suggala, and Pradeep Ravikumar. 2015. “Vector-Space Markov Random Fields via Exponential Families.” In Journal of Machine Learning Research, 684–92. http://jmlr.csail.mit.edu/proceedings/papers/v37/tansey15.html.

Vajda, S. 1951. “Analytical Studies in Stop-Loss Reinsurance.” Scandinavian Actuarial Journal 1951 (1-2): 158–75. https://doi.org/10.1080/03461238.1951.10432137.

Wainwright, Martin J., and Michael I. Jordan. 2008. Graphical Models, Exponential Families, and Variational Inference. Vol. 1. Foundations and Trends® in Machine Learning. http://www.cs.berkeley.edu/~jordan/papers/wainwright-jordan-fnt.pdf.