Generically approximating probability distributions
2021-03-11 — 2021-03-22
Wherein several methods of approximating probability laws are exhibited, Edgeworth expansions and kernel, empirical, and variational approximations are presented, and closeness is measured in probability metrics.
There are various approximations we might use for a probability distribution. Empirical CDFs, kernel density estimates, variational approximation, Edgeworth expansions, Laplace approximations…
From each of these, we might get close in some metric to the desired target.
This is a broad topic that I cannot hope to cover in full generality. So I made a landing page to remind me of the many methods that I forget. Special cases of interest include:
- Statements about where the probability mass is with high probability (concentration theorems)
- Statements about the asymptotic distributions of variables eventually approaching some distribution as some parameter goes to infinity (limit theorems). Most famously, a lot of things approach normal distributions, but there are other limit theorems.
There are other types of results besides these in this domain. I am interested in collecting results that tell me about how various combinations of variables approach a limiting distribution in some probability metric.
1 Stein’s method
See Stein’s method.