# Learning on manifolds

Finding the lowest bit of a krazy straw, from the inside

October 21, 2011 — January 26, 2022

A placeholder for learning on curved spaces. Not discussed: learning OF the curvature of spaces.

AFAICT this usually boils down to defining an appropriate stochastic process on a manifold.

## 1 Learning on a given manifold

Learning where there is an *a priori* manifold seems to also be a usage here? For example the manifold of positive definite matrices is treated in depth in Chikuse and 筑瀬 (2003).

See the work of, e.g.

Manifold optimisation implementations:

- pytorch: Lezcano/geotorch: Constrained optimization toolkit for PyTorch (Lezcano Casado 2019)
- MATLAB: manopt,
- Python: pymanopt.
- Julia: Manopt.jl
- Python: Nina Miolane et al’s Geomstats project.
- C++: ROPTLIB (Huang et al. 2018)
- R: ManifoldOptim wrapes ROPTLIB (Martin et al. 2016)

There are at least two textbooks online:

## 2 Information Geometry

The unholy offspring of Fisher information and differential geometry, about which I know little except that it sounds like it should be intuitive. It is probably synonymous with some of the other items on this page if I could sort out all this terminology. See information geometry.

## 3 Hamiltonian Monte Carlo

You can also discuss Hamiltonian Monte Carlo in this setting. I will not.

## 4 Langevin Monte Carlo

Girolami et al discuss Langevin Monte Carlo in this context.

## 5 Natural gradient

See natural gradients.

## 6 Homogeneous probability

Albert Tarantola’s framing, from his manuscript. How does it relate to information geometry? I don’t know yet. Haven’t had time to read. Also not a common phrasing, which is a danger sign.

## 7 Incoming

- Agustinus Kristiadi, Fisher Information Matrix
- Agustinus Kristiadi, Hessian and Curvatures in Machine Learning: A Differential-Geometric View
- Agustinus Kristiadi, Notes on Riemannian Geometry
- Agustinus Kristiadi, Optimization and Gradient Descent on Riemannian Manifolds

## 8 References

*Optimization algorithms on matrix manifolds*.

*Differential Geometry in Statistical Inference*.

*Neural Computation*.

*IEEE Transactions on Information Theory*.

*The Annals of Statistics*.

*Differential Geometry in Statistical Inference*.

*Bernoulli*.

*arXiv:2006.10160 [Cs, Stat]*.

*IEEE Transactions on Signal Processing*.

*Journal of Machine Learning Research*.

*Information and Inference*.

*International Journal of Computer Vision*.

*IEEE Transactions on Signal Processing*.

*Statistics on Special Manifolds*.

*Journal of Applied Geophysics*.

*Advances In Neural Information Processing Systems*.

*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*.

*arXiv:2104.05508 [Cs, Stat]*.

*arXiv Preprint arXiv:1506.07677*.

*ACM Transactions on Mathematical Software*.

*Differential Geometry in Statistical Inference*.

*Annual Review of Statistics and Its Application*.

*Advances in Neural Information Processing Systems*.

*IEEE Journal of Selected Topics in Signal Processing*.

*Directional Statistics*.

*arXiv:1805.08308 [Cs, Stat]*.

*Journal of Geophysical Research: Solid Earth*.

*Bernoulli*.

*Scholarpedia*.

*Transactions on Machine Learning Research*.

*Proceedings of the National Academy of Sciences*.

*Journal of Applied Probability*.

*Advances in Neural Information Processing Systems 21*.

*Journal of Machine Learning Research*.

*Physical Review E*.

*arXiv:1608.04026 [Math]*.

*Statistics & Probability Letters*.