Learning on manifolds
Finding the lowest bit of a krazy straw, from the inside
October 21, 2011 — January 26, 2022
A placeholder for learning on curved spaces. Not discussed: learning OF the curvature of spaces.
AFAICT this usually boils down to defining an appropriate stochastic process on a manifold.
1 Learning on a given manifold
Learning where there is an a priori manifold seems to also be a usage here? For example, the manifold of positive definite matrices is treated in depth in Chikuse and 筑瀬 (2003).
See the work of, e.g.
Manifold optimisation implementations:
- pytorch: Lezcano/geotorch: Constrained optimization toolkit for PyTorch (Lezcano Casado 2019)
- MATLAB: manopt
- Python: pymanopt
- Julia: Manopt.jl
- Python: Nina Miolane et al’s Geomstats project
- C++: ROPTLIB (Huang et al. 2018)
- R: ManifoldOptim wraps ROPTLIB (Martin et al. 2016)
There are at least two textbooks online:
2 Information Geometry
The unholy offspring of Fisher information and differential geometry, about which I know little except that it sounds like it should be intuitive. It is probably synonymous with some of the other items on this page if I could sort out all this terminology. See information geometry.
3 Hamiltonian Monte Carlo
You can also discuss Hamiltonian Monte Carlo in this setting. I will not.
4 Langevin Monte Carlo
Girolami et al discuss Langevin Monte Carlo in this context.
5 Natural gradient
See natural gradients.
6 Homogeneous probability
Albert Tarantola’s framing, from his manuscript. How does it relate to information geometry? I don’t know yet. Haven’t had time to read. Also not a common phrasing, which is a danger sign.
7 Incoming
Agustinus Kristiadi, Fisher Information Matrix
Agustinus Kristiadi, Hessian and Curvatures in Machine Learning: A Differential-Geometric View
Agustinus Kristiadi, Notes on Riemannian Geometry
Agustinus Kristiadi, Optimization and Gradient Descent on Riemannian Manifolds
GPflow/GeometricKernels: Geometric kernels on manifolds, meshes, and graphs
pyg-team/pytorch_geometric: Graph Neural Network Library for PyTorch
Peter Woit, Symmetry and Physics
It’s getting late, but I can’t help myself. Reading too many wrong things about symmetry and physics on Twitter has forced me to do this. And, John Baez says I don’t explain things. So, here’s what the relationship between symmetry and physics really is.
Riemannian Score-Based Generative Modelling/ oxcsml/riemannian-score-sde: Score-based generative models for compact manifolds