Probability

July 29, 2011 — January 18, 2025

algebra
functional analysis
probability

Suspiciously similar content

Figure 1

A placeholder where I mention some things I occasionally refer people to or have been referred to, but forget the reference for.

1 What even is probability theory about?

It’s about models for regularities in how events occur. In Bayesian terms, probability is about how we should update our beliefs about the world when we see new things happen, given some model for how things happen. In frequentist terms, it’s about the distribution of outcomes of experiments with some kind of regularity of distribution (usually conditional dependence). There are other models too, but I know them less well.

Neither of those is “directly” “about” the world, but it seems we can do very well by using either of them as a model for the world, so maybe they are pretty good models for the world. This gap between the world and this model of it leads some to claim that probability does not “exist” (Nau 2001; Spiegelhalter 2024), which is technically true under a sufficiently restrictive definition of “exist”, but not one that I use in day-to-day life.

BTW I wrote and animated an introductory lecture about this way of explaining probability.

2 Probability hacks

Figure 2: Tyche, roller of dice and collapser of waveforms.

3 Rényi axioms

Founded on conditional probability (Mečíř 2020; Taraldsen 2019) which “feels” more natural to me, but YMMV.

4 Cox probability

Also founded on conditional probability, but more Bayesey I think. See Cox probability

5 References

Aldous. 1981. “Representations for Partially Exchangeable Arrays of Random Variables.” Journal of Multivariate Analysis.
———. 1985. “Exchangeability and Related Topics.” In École d’Été de Probabilités de Saint-Flour XIII — 1983. Lecture Notes in Mathematics.
Applebaum. 2009. Lévy Processes and Stochastic Calculus. Cambridge Studies in Advanced Mathematics 116.
Bao, and Ullah. 2009. “Expectation of Quadratic Forms in Normal and Nonnormal Variables with Econometric Applications.” 200907. Working Papers.
Baudoin. 2014. Diffusion Processes and Stochastic Calculus. EMS Textbooks in Mathematics.
Burgess. 2014. “Martingale Measures & Change of Measure Explained.” SSRN Scholarly Paper ID 2961006.
Campbell, Syed, Yang, et al. 2019. “Local Exchangeability.” arXiv:1906.09507 [Math, Stat].
Cosma Rohilla Shalizi. 2007. Almost none of the theory of stochastic processes.
Diaconis, and Freedman. 1980. “De Finetti’s Theorem for Markov Chains.” The Annals of Probability.
Fleming, and Harrington. 2005. “Appendix A: Some Results from Stieltjes Integration and Probability Theory.” In Counting Processes and Survival Analysis.
Gray. 1987. Probability, Random Processes, and Ergodic Properties.
Grinstead, and Snell. 1997. Introduction to Probability.
Hanson, and Wright. 1971. “A Bound on Tail Probabilities for Quadratic Forms in Independent Random Variables.” Annals of Mathematical Statistics.
Jacod, and Shiryaev. 1987. Limit Theorems for Stochastic Processes. Grundlehren Der Mathematischen Wissenschaften.
Kallenberg. 2017. Random Measures, Theory and Applications.
Mathai, and Provost. 1992. Quadratic Forms in Random Variables: Theory and Applications. Statistics, Textbooks and Monographs, v. 126.
Mečíř. 2020. “Foundations for Conditional Probability.”
Nau. 2001. “De Finetti Was Right: Probability Does Not Exist.” Theory and Decision.
Peng. 2007. “G-Expectation, G-Brownian Motion and Related Stochastic Calculus of Itô Type.” In Stochastic Analysis and Applications.
Pitman, and Yor. n.d. “A Guide to Brownian Motion and Related Stochastic Processes.”
Rényi. 1970. Foundations of Probability.
Resnick. 1992. Adventures in Stochastic Processes.
Soch, Proofs, Faulkenberry, et al. 2020. “StatProofBook/StatProofBook.github.io: StatProofBook 2020.”
Spiegelhalter. 2024. “Why Probability Probably Doesn’t Exist (but It Is Useful to Act Like It Does).” Nature.
Taraldsen. 2019. “Conditional Probability in Rényi Spaces.”
van Zanten. 2004. An introduction to stochastic processes in continuous time.
Wright. 1973. “A Bound on Tail Probabilities for Quadratic Forms in Independent Random Variables Whose Distributions Are Not Necessarily Symmetric.” Annals of Probability.