# Bootstrap

Shuffling reality to produce your data

November 26, 2014 — January 27, 2022

estimator distribution
nonparametric
probabilistic algorithms
statistics
uncertainty

Resampling your own data to estimate how good your point-estimator is, and to reduce its bias. In general an intuitive technique. However, gets tricky for e.g. dependent data. For a handy crib sheet for bootstrap failure modes, see Thomas Lumley, When the bootstrap doesn’t work.

In the classical mode, this is a frequentist technique without an immediate Bayesian interpretation.

Commonly credited as being invented by B. Efron (1979) and theoretically justified by Gine and Zinn (1990).

## 2 Bootstrap bias correction

As opp variance estimation. NBD; Bootstrap is notionally telling you the sampling distribution. 🏗

## 3 Bootstrap for dependent data

e.g., as presaged, time series. Parametric bootstrap would be the logical default choice, right? When does that work?

Now a thing!

## 5 As a Bayesian method

There is absolutely a Bayesian bootstrap if you think hard enough about it, it turns out. Several, really. Rubin (1981) derived a Bayesian version. See Lyddon, Holmes, and Walker (2019) for a modern update, and Rasmus Bååth for a diagrammed explanation of the points of contact with frequentist bootstrap and some other things.

## 6 Pedagogic

• software: boot for R

• Tim Hesterberg’s teaching notes:

## 7 References

Alfaro. 2003. Molecular Biology and Evolution.
Bach. 2009. arXiv:0901.3202 [Cs, Stat].
Barber, Candès, Ramdas, et al. 2021. The Annals of Statistics.
Biewen. 2002. Journal of Econometrics.
Bühlmann. 2002. Statistical Science.
Bühlmann, and Künsch. 1999. Computational Statistics & Data Analysis.
Burnham, and Anderson. 2004. Sociological Methods & Research.
Chang, and Hall. 2015. Biometrika.
Chen, and Lo. 1997. Probability Theory and Related Fields.
Cogneau, and Zakamouline. 2010.
Dahlhaus. 2011. Journal of the Korean Statistical Society.
DiCiccio, and Efron. 1996. Statistical Science.
Efron, B. 1979. The Annals of Statistics.
———. 2012. The Annals of Applied Statistics.
———. 2021. Stats.
Fong, and Holmes. 2020. Biometrika.
Galvani, Bardelli, Figini, et al. 2021. Algorithms.
Gine, and Zinn. 1990. Annals of Probability.
Giordano, Jordan, and Broderick. 2019. arXiv:1907.12116 [Cs, Math, Stat].
Gonçalves, and Politis. 2011. Journal of the Korean Statistical Society.
Gonçalves, and White. 2004. Journal of Econometrics.
Good, and Good. 1999. Resampling Methods: A Practical Guide to Data Analysis.
Götze, and Künsch. 1996. The Annals of Statistics.
Green, and Shalizi. 2017. arXiv:1711.00813 [Stat].
Hall. 1992. The Annals of Statistics.
———. 1994. In Handbook of Econometrics.
Hall, Horowitz, and Jing. 1995. Biometrika.
Härdle, Horowitz, and Kreiss. 2003. International Statistical Review.
Hesterberg. 2011. Bootstrap.” Wiley Interdisciplinary Reviews: Computational Statistics.
Hinkley. 1997. Bootstrap Methods and Their Application.
Imbens, and Menzel. 2021. The Annals of Statistics.
Künsch. 1989. The Annals of Statistics.
Lahiri. 1993. Statistics & Probability Letters.
———. 2001. Probability Theory and Related Fields.
———. 2003. Resampling Methods for Dependent Data.
Lee, and Young. 1996. Statistical Science.
Lyddon, Holmes, and Walker. 2019. Biometrika.
Papadopoulos, Edwards, and Murray. 2001. IEEE Transactions on Neural Networks.
Paparoditis, and Sapatinas. 2014. arXiv:1409.4317 [Math, Stat].
Politis. 2003. Statistical Science.
Politis, and Romano. 1994. Journal of the American Statistical Association.
Politis, and White. 2004. Econometric Reviews.
Rodriguez, and Ruiz. 2009. Journal of Time Series Analysis.
Rubin. 1981. Annals of Statistics.
Sanson, Strange, and Garry. 2019. Clinical Psychological Science.
Shalizi. 2010. American Scientist.
Shao. 1996. Journal of the American Statistical Association.
Shibata. 1997. “Bootstrap Estimate of Kullback-Leibler Information for Model Selection.” Statistica Sinica.
Stone. 1977. Journal of the Royal Statistical Society. Series B (Methodological).
Tibshirani, Rinaldo, Tibshirani, et al. 2015. arXiv:1506.06266 [Math, Stat].
Vogel, and Shallcross. 1996. Water Resources Research.
Yatchew, and Hardle. 2006. “Nonparametric State Price Density Estimation Using Constrained Least Squares and the Bootstrap.” Journal of Econometrics.