Generically approximating probability distributions


There are various approximations we might use for a a probability distribution. Empirical CDFs, Kernel density estimates, variational approximation, Edgeworth expansions, Laplace approximations…

From each of these we might get close in some metric to the desired target.

This is a broad topic which I cannot hope to cover in full generality. Special cases of interest include

  1. Statements about where the probability mass is with high probability (concentration theorems)
  2. statements about the asymptotic distributions of variables eventually approaching some distribution as some parameter goes to infinity (limit theorems. Most famously a lot of things approach normal distributions, but there are many limit theorems

There are other types of results besides, in this domain. I am quite interested in collecting here, some results that tell me about how various combinations of variables approach a limiting distribution in some probability metric.

Stein’s method

See Stein’s method.

References

Chatterjee, Sourav, and Elizabeth Meckes. 2008. “Multivariate Normal Approximation Using Exchangeable Pairs.” January 15, 2008. http://arxiv.org/abs/math/0701464.
Meckes, Elizabeth. 2006. “An Infinitesimal Version of Stein’s Method of Exchangeable Pairs.”
———. 2009. “On Stein’s Method for Multivariate Normal Approximation.” In High Dimensional Probability V: The Luminy Volume, 153–78. Beachwood, Ohio, USA: Institute of Mathematical Statistics. https://doi.org/10.1214/09-IMSCOLL511.
Reinert, Gesine, and Adrian Röllin. 2007. “Multivariate Normal Approximation with Stein’s Method of Exchangeable Pairs Under a General Linearity Condition,” November. https://doi.org/10.1214/09-AOP467.
Stein, Charles. 1972. “A Bound for the Error in the Normal Approximation to the Distribution of a Sum of Dependent Random Variables.” Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Probability Theory, January, 583–602. https://projecteuclid.org/ebooks/berkeley-symposium-on-mathematical-statistics-and-probability/Proceedings-of-the-Sixth-Berkeley-Symposium-on-Mathematical-Statistics-and/chapter/A-bound-for-the-error-in-the-normal-approximation-to/bsmsp/1200514239.
———. 1986. Approximate Computation of Expectations. Vol. 7. IMS. https://www.jstor.org/stable/4355512.
Vershynin, Roman. 2018. High-Dimensional Probability: An Introduction with Applications in Data Science. 1st ed. Cambridge University Press. https://doi.org/10.1017/9781108231596.

Warning! Experimental comments system! If is does not work for you, let me know via the contact form.

No comments yet!

GitHub-flavored Markdown & a sane subset of HTML is supported.