Learning on manifolds
Finding the lowest bit of a krazy straw, from the inside
October 21, 2011 — January 26, 2022
A placeholder for learning on curved spaces. Not discussed: learning OF the curvature of spaces.
AFAICT this usually boils down to defining an appropriate stochastic process on a manifold.
1 Learning on a given manifold
Learning where there is an a priori manifold seems to also be a usage here? For example the manifold of positive definite matrices is treated in depth in Chikuse and 筑瀬 (2003).
See the work of, e.g.
Manifold optimisation implementations:
- pytorch: Lezcano/geotorch: Constrained optimization toolkit for PyTorch (Lezcano Casado 2019)
- MATLAB: manopt,
- Python: pymanopt.
- Julia: Manopt.jl
- Python: Nina Miolane et al’s Geomstats project.
- C++: ROPTLIB (Huang et al. 2018)
- R: ManifoldOptim wrapes ROPTLIB (Martin et al. 2016)
There are at least two textbooks online:
2 Information Geometry
The unholy offspring of Fisher information and differential geometry, about which I know little except that it sounds like it should be intuitive. It is probably synonymous with some of the other items on this page if I could sort out all this terminology. See information geometry.
3 Hamiltonian Monte Carlo
You can also discuss Hamiltonian Monte Carlo in this setting. I will not.
4 Langevin Monte Carlo
Girolami et al discuss Langevin Monte Carlo in this context.
5 Natural gradient
See natural gradients.
6 Homogeneous probability
Albert Tarantola’s framing, from his manuscript. How does it relate to information geometry? I don’t know yet. Haven’t had time to read. Also not a common phrasing, which is a danger sign.
7 Incoming
- Agustinus Kristiadi, Fisher Information Matrix
- Agustinus Kristiadi, Hessian and Curvatures in Machine Learning: A Differential-Geometric View
- Agustinus Kristiadi, Notes on Riemannian Geometry
- Agustinus Kristiadi, Optimization and Gradient Descent on Riemannian Manifolds