Information geometry

A placeholder for a particular type of learning on curved spaces about which I do not in fact know anything.

Maybe see Azimuth’s Information Geometry Series plus the overview


Amari, Shun-ichi. 1998. Natural Gradient Works Efficiently in Learning.” Neural Computation 10 (2): 251–76.
Amari, Shun-ichi, Ryo Karakida, and Masafumi Oizumi. 2018. Fisher Information and Natural Gradient Learning of Random Deep Networks.” arXiv:1808.07172 [Cond-Mat, Stat], August.
Amari, Shun-ichi, Hyeyoung Park, and Kenji Fukumizu. 2000. Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons.” Neural Computation 12 (6): 1399–409.
Amari, Shunʼichi. 1987. “Differential Geometrical Theory of Statistics.” In Differential Geometry in Statistical Inference, 19–94.
———. 2001. Information Geometry on Hierarchy of Probability Distributions.” IEEE Transactions on Information Theory 47: 1701–11.
Ay, Nihat. 2002. An Information-Geometric Approach to a Theory of Pragmatic Structuring.” The Annals of Probability 30 (1): 416–36.
Barndorff-Nielsen, O E. 1987. “Differential and Integral Geometry in Statistical Inference.” In Differential Geometry in Statistical Inference. Sn Aarhus.
Brody, Dorje, and Nicolas Rivier. 1995. Geometrical Aspects of Statistical Mechanics.” Phys. Rev. E 51 (2): 1006–11.
Csiszar, I. 1975. I-Divergence Geometry of Probability Distributions and Minimization Problems.” Annals of Probability 3 (1): 146–58.
Doyle, Peter G., and Jean Steiner. 2011. Commuting Time Geometry of Ergodic Markov Chains,” July.
Fernández-Martínez, J. L., Z. Fernández-Muñiz, J. L. G. Pallero, and L. M. Pedruelo-González. 2013. From Bayes to Tarantola: New Insights to Understand Uncertainty in Inverse Problems.” Journal of Applied Geophysics 98 (November): 62–72.
Kass, Robert E., Shun-Ichi Amari, Kensuke Arai, Emery N. Brown, Casey O. Diekman, Markus Diesmann, Brent Doiron, et al. 2018. Computational Neuroscience: Mathematical and Statistical Perspectives.” Annual Review of Statistics and Its Application 5 (1): 183–214.
Khan, Gabriel, and Jun Zhang. 2022. When Optimal Transport Meets Information Geometry.” Information Geometry, June.
Kulhavý, Rudolf. 1990. Recursive Nonlinear Estimation: A Geometric Approach.” Automatica 26 (3): 545–55.
Lauritzen, S L. 1987. “Statistical Manifolds.” In Differential Geometry in Statistical Inference, 164. JSTOR.
Ly, Alexander, Maarten Marsman, Josine Verhagen, Raoul P. P. P. Grasman, and Eric-Jan Wagenmakers. 2017. A Tutorial on Fisher Information.” Journal of Mathematical Psychology 80 (October): 40–55.
Mallasto, Anton, Augusto Gerolin, and Hà Quang Minh. 2021. Entropy-Regularized 2-Wasserstein Distance Between Gaussian Measures.” Information Geometry, August.
Martens, James. 2020. New Insights and Perspectives on the Natural Gradient Method.” Journal of Machine Learning Research 21 (146): 1–76.
Miolane, Nina, Johan Mathe, Claire Donnat, Mikael Jorda, and Xavier Pennec. 2018. Geomstats: A Python Package for Riemannian Geometry in Machine Learning.” arXiv:1805.08308 [Cs, Stat], May.
Mosegaard, Klaus, and Albert Tarantola. 1995. Monte Carlo Sampling of Solutions to Inverse Problems.” Journal of Geophysical Research: Solid Earth 100 (B7): 12431–47.
———. 2002. Probabilistic Approach to Inverse Problems.” In International Geophysics, 81:237–65. Elsevier.
Nielsen, Frank. 2018. An Elementary Introduction to Information Geometry.” arXiv:1808.08271 [Cs, Math, Stat], August.
Palomar, D.P., and S. Verdu. 2008. Lautum Information.” IEEE Transactions on Information Theory 54 (3): 964–75.
Poole, Ben, Subhaneil Lahiri, Maithreyi Raghu, Jascha Sohl-Dickstein, and Surya Ganguli. 2016. Exponential Expressivity in Deep Neural Networks Through Transient Chaos.” In Advances in Neural Information Processing Systems 29, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 3360–68. Curran Associates, Inc.
Raginsky, Maxim, and Igal Sason. 2012. Concentration of Measure Inequalities in Information Theory, Communications and Coding.” Foundations and Trends in Communications and Information Theory, December.
Transtrum, Mark K, Benjamin B Machta, and James P Sethna. 2011. The Geometry of Nonlinear Least Squares with Applications to Sloppy Models and Optimization.” Physical Review E 83 (3): 036701.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.