Ergodic theory / mixing

For things that probably happen eventually instead of just probably


Relevance to actual stochastic processes and dynamical systems, especially linear and non-linear system identification.

Keywords to look up:

  • probability-free ergodicity
  • Birkhoff ergodic theorem
  • Frobenius-Perron operator
  • Quasicompactness, correlation decay
  • C&C CLT for Markov chains – Nagaev
  • Coupling from the past

Not much material here, but please see learning theory for dependent data for some interesting categorisations of mixing and transcendence of miscellaneous mixing conditions for statistical estimators.

Coupling from the past

Dan Piponi does a functional programming explanation of coupling from the past for markov chains.

Mixing zoo






Sequential Rademacher complexity


Brémaud, Pierre, and Laurent Massoulié. 2001. “Hawkes Branching Point Processes Without Ancestors.” Journal of Applied Probability 38 (1): 122–35.

Delft, Anne van, and Michael Eichler. 2016. “Locally Stationary Functional Time Series,” February.

Diaconis, Persi, and David Freedman. 1999. “Iterated Random Functions.” SIAM Review 1 (1): 45–76.

Gray, Robert M. 2009. Probability, Random Processes, and Ergodic Properties. Springer Verlag.

Keane, Michael, and Karl Petersen. 2006. “Easy and Nearly Simultaneous Proofs of the Ergodic Theorem and Maximal Ergodic Theorem.” IMS Lecture Notes-Monograph Series Dynamics & Stochastics 48.

Kuznetsov, Vitaly, and Mehryar Mohri. 2016. “Generalization Bounds for Non-Stationary Mixing Processes.” In Machine Learning Journal.

———. 2014. “Generalization Bounds for Time Series Prediction with Non-Stationary Processes.” In Algorithmic Learning Theory, edited by Peter Auer, Alexander Clark, Thomas Zeugmann, and Sandra Zilles, 260–74. Lecture Notes in Computer Science. Bled, Slovenia: Springer International Publishing.

Livan, Giacomo, Jun-ichi Inoue, and Enrico Scalas. 2012. “On the Non-Stationarity of Financial Time Series: Impact on Optimal Portfolio Selection.” Journal of Statistical Mechanics: Theory and Experiment 2012 (07): P07025.

McDonald, Daniel J., Cosma Rohilla Shalizi, and Mark Schervish. 2011. “Risk Bounds for Time Series Without Strong Mixing,” June.

Mohri, Mehryar, and Afshin Rostamizadeh. 2009. “Stability Bounds for Stationary ϕ-Mixing and β-Mixing Processes.” Journal of Machine Learning Research 4: 1–26.

Morvai, Gusztáv, Sidney Yakowitz, and László Györfi. 1996. “Nonparametric Inference for Ergodic, Stationary Time Series.” The Annals of Statistics 24 (1): 370–79.

Palmer, Richard G. 1982. “Broken Ergodicity.” Advances in Physics 31 (6): 669–735.

Propp, James Gary, and David Bruce Wilson. 1996. “Exact Sampling with Coupled Markov Chains and Applications to Statistical Mechanics.” In Random Structures & Algorithms, 9:223–52. New York, NY, USA: John Wiley & Sons, Inc.<223::AID-RSA14>3.0.CO;2-O.

———. 1998. “Coupling from the Past: A User’s Guide.” In Microsurveys in Discrete Probability, edited by David Aldous and James Gary Propp, 41:181–92. DIMACS Series in Discrete Mathematics and Theoretical Computer Science. Providence, Rhode Island: American Mathematical Society.

Rosenblatt, M. 1984. “Asymptotic Normality, Strong Mixing and Spectral Density Estimates.” The Annals of Probability 12 (4): 1167–80.

Ryabko, Daniil, and Boris Ryabko. 2010. “Nonparametric Statistical Inference for Ergodic Processes.” IEEE Transactions on Information Theory 56 (3): 1430–5.

Shao, Xiaofeng, and Wei Biao Wu. 2007. “Asymptotic Spectral Theory for Nonlinear Time Series.” The Annals of Statistics 35 (4): 1773–1801.

Shields, P C. 1998. “The Interactions Between Ergodic Theory and Information Theory.” IEEE Transactions on Information Theory 44 (6): 2079–93.

Steif, Jeffrey E. 1997. “Consistent Estimation of Joint Distributions for Sufficiently Mixing Random Fields.” The Annals of Statistics 25 (1): 293–304.

Stein, D L, and C M Newman. 1995. “Broken Ergodicity and the Geometry of Rugged Landscapes.” Physical Review E 51 (6): 5228–38.

Thouvenot, Jean-Paul, and Benjamin Weiss. 2012. “Limit Laws for Ergodic Processes.” Stochastics and Dynamics 12 (01): 1150012.