Bregman divergences
2023-08-29 — 2023-08-29
Wherein a measure of separation between points is presented, being defined from a strictly convex function, exemplified by the squared Euclidean distance, and being employed in mirror descent.
In mathematics, specifically statistics and information geometry, a Bregman divergence or Bregman distance is a measure of difference between two points, defined in terms of a strictly convex function. They form an important class of divergences. When the points are interpreted as probability distributions — notably as either values of the parameter of a parametric model or as a data set of observed values — the resulting distance is a statistical distance. The most basic Bregman divergence is the squared Euclidean distance.
Useful in mirror descent.
- Meet the Bregman Divergences ← Inductio Ex Machina ← Mark Reid
- Bregman divergences, dual information geometry, and generalized convexity
- connection to score matching, contrastive divergence, mirror descent