Bootstrap

2014-11-26 – 2019-09-09

Resampling your own data to estimate how good your estimator is, frequentist-style. Gets tricky for dependent data, and various other things.

For a handy crib sheet for bootstrap failure modes, see Thomas Lumley, When the bootstrap doesn’t work.

Bootstrap bias correction

As opp variance estimation. When does this work? 🏗

Boostrap for dependent data

e.g., as presaged, time series. Parametric bootstrap should work for this, right?

“Wild bootstrap”

What is this even? 🏗

Pedagogic

Bach, Francis. 2009. “Model-Consistent Sparse Estimation Through the Bootstrap.” arXiv:0901.3202 [Cs, Stat]. https://hal.archives-ouvertes.fr/hal-00354771/document.

Biewen, Martin. 2002. “Bootstrap Inference for Inequality, Mobility and Poverty Measurement.” Journal of Econometrics 108 (2): 317–42. https://doi.org/10.1016/S0304-4076(01)00138-5.

Bühlmann, Peter. 2002. “Bootstraps for Time Series.” Statistical Science 17 (1): 52–72. ftp://stat.ethz.ch/Research-Reports/87.pdf.

Bühlmann, Peter, and Hans R Künsch. 1999. “Block Length Selection in the Bootstrap for Time Series.” Computational Statistics & Data Analysis 31 (3): 295–310. https://doi.org/10.1016/S0167-9473(99)00014-6.

Chang, Jinyuan, and Peter Hall. 2015. “Double-Bootstrap Methods That Use a Single Double-Bootstrap Simulation.” Biometrika 102 (1): 203–14. https://doi.org/10.1093/biomet/asu060.

Chen, Kani, and Shaw-Hwa Lo. 1997. “On a Mapping Approach to Investigating the Bootstrap Accuracy.” Probability Theory and Related Fields 107 (2): 197–217. https://doi.org/10.1007/s004400050083.

Cogneau, Philippe, and Valeri Zakamouline. 2010. “Bootstrap Methods for Finance: Review and Analysis.” Working Paper, University of Agder. http://www.seminar.hec.ulg.ac.be/docs/Sem21.10.10_Cogneau.pdf.

Dahlhaus, Rainer. 2011. “Discussion: Bootstrap Methods for Dependent Data: A Review.” Journal of the Korean Statistical Society 40 (4): 379–81. https://doi.org/10.1016/j.jkss.2011.07.004.

DiCiccio, Thomas J., and Bradley Efron. 1996a. “Bootstrap Confidence Intervals.” Statistical Science 11 (3): 189–212. https://doi.org/10.1214/ss/1032280214.

———. 1996b. “[Bootstrap Confidence Intervals]: Rejoinder.” Statistical Science 11 (3): 223–28. http://www.jstor.org/stable/2246115.

Efron, B. 1979. “Bootstrap Methods: Another Look at the Jackknife.” The Annals of Statistics 7 (1): 1–26. https://doi.org/10.1214/aos/1176344552.

Efron, Bradley. 2012. “Bayesian Inference and the Parametric Bootstrap.” The Annals of Applied Statistics 6 (4): 1971–97. https://doi.org/10.1214/12-AOAS571.

———. 1981. “Nonparametric Estimates of Standard Error: The Jackknife, the Bootstrap and Other Methods.” Biometrika 68 (3): 589–99. https://doi.org/10.1093/biomet/68.3.589.

Giordano, Ryan, Michael I. Jordan, and Tamara Broderick. 2019. “A Higher-Order Swiss Army Infinitesimal Jackknife,” July. http://arxiv.org/abs/1907.12116.

Gonçalves, Sílvia, and Dimitris Politis. 2011. “Discussion: Bootstrap Methods for Dependent Data: A Review.” Journal of the Korean Statistical Society 40 (4): 383–86. https://doi.org/10.1016/j.jkss.2011.07.003.

Gonçalves, Sílvia, and Halbert White. 2004. “Maximum Likelihood and the Bootstrap for Nonlinear Dynamic Models.” Journal of Econometrics 119 (1): 199–219. https://doi.org/10.1016/S0304-4076(03)00204-5.

Götze, F., and H. R. Künsch. 1996. “Second-Order Correctness of the Blockwise Bootstrap for Stationary Observations.” The Annals of Statistics 24 (5): 1914–33. https://doi.org/10.1214/aos/1069362303.

Green, Alden, and Cosma Rohilla Shalizi. 2017. “Bootstrapping Exchangeable Random Graphs,” November. http://arxiv.org/abs/1711.00813.

Hall, Peter. 1992. “On Bootstrap Confidence Intervals in Nonparametric Regression.” The Annals of Statistics 20 (2): 695–711. http://www.jstor.org/stable/2241979.

———. 1994. “Chapter 39 Methodology and Theory for the Bootstrap.” In, edited by BT - Handbook of Econometrics, 4:2341–81. Elsevier. http://www.sciencedirect.com/science/article/pii/S157344120580008X.

Hall, Peter, Joel L. Horowitz, and Bing-Yi Jing. 1995. “On Blocking Rules for the Bootstrap with Dependent Data.” Biometrika 82 (3): 561–74. https://doi.org/10.1093/biomet/82.3.561.

Härdle, Wolfgang, Joel Horowitz, and Jens-Peter Kreiss. 2003. “Bootstrap Methods for Time Series.” International Statistical Review 71 (2): 435–59. http://onlinelibrary.wiley.com/doi/10.1111/j.1751-5823.2003.tb00485.x/abstract.

Hesterberg, Tim. 2011. “Bootstrap.” Wiley Interdisciplinary Reviews: Computational Statistics 3 (6): 497–526. https://doi.org/10.1002/wics.182.

Hinkley, David V. 1997. Bootstrap Methods and Their Application. Cambridge ; New York, NY, USA: Cambridge University Press. http://www.loc.gov/catdir/description/cam027/96030064.html.

Künsch, Hans Rudolf. 1989. “The Jackknife and the Bootstrap for General Stationary Observations.” The Annals of Statistics 17 (3): 1217–41. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.28.924&rep=rep1&type=pdf.

Lahiri, S N. 1993. “On the Moving Block Bootstrap Under Long Range Dependence.” Statistics & Probability Letters 18 (5): 405–13. https://doi.org/10.1016/0167-7152(93)90035-H.

———. 2001. “Effects of Block Lengths on the Validity of Block Resampling Methods.” Probability Theory and Related Fields 121: 73–97. https://doi.org/10.1007/PL00008798.

———. 2003. Resampling Methods for Dependent Data. New York: Springer.

Lee, Stephen M. S., and G. Alastair Young. 1996. “[Bootstrap Confidence Intervals]: Comment.” Statistical Science 11 (3): 221–23. http://www.jstor.org/stable/2246114.

Paparoditis, Efstathios, and Theofanis Sapatinas. 2014. “Bootstrap-Based Testing for Functional Data,” September. http://arxiv.org/abs/1409.4317.

Politis, Dimitris N. 2003. “The Impact of Bootstrap Methods on Time Series Analysis.” Statistical Science 18 (2): 219–30. https://doi.org/10.1214/ss/1063994977.

Politis, Dimitris N., and Joseph P. Romano. 1994. “The Stationary Bootstrap.” Journal of the American Statistical Association 89 (428): 1303–13. https://doi.org/10.1080/01621459.1994.10476870.

Politis, Dimitris N., and Halbert White. 2004. “Automatic Block-Length Selection for the Dependent Bootstrap.” Econometric Reviews 23 (1): 53–70. https://doi.org/10.1081/ETC-120028836.

Rodriguez, Alejandro, and Esther Ruiz. 2009. “Bootstrap Prediction Intervals in State–Space Models.” Journal of Time Series Analysis 30 (2): 167–78. https://doi.org/10.1111/j.1467-9892.2008.00604.x.

Shalizi, Cosma Rohilla. 2010. “The Bootstrap.” American Scientist 98 (3): 186. https://doi.org/10.1511/2010.84.186.

Shao, Jun. 1996. “Bootstrap Model Selection.” Journal of the American Statistical Association 91 (434): 655–65. https://doi.org/10.2307/2291661.

Shibata, Ritei. 1997. “Bootstrap Estimate of Kullback-Leibler Information for Model Selection.” Statistica Sinica 7: 375–94.

Stone, M. 1977. “An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike’s Criterion.” Journal of the Royal Statistical Society. Series B (Methodological) 39 (1): 44–47. http://www.stat.washington.edu/courses/stat527/s14/readings/Stone1977.pdf.

Tibshirani, Ryan J., Alessandro Rinaldo, Robert Tibshirani, and Larry Wasserman. 2015. “Uniform Asymptotic Inference and the Bootstrap After Model Selection,” June. http://arxiv.org/abs/1506.06266.

Vogel, Richard M., and Amy L. Shallcross. 1996. “The Moving Blocks Bootstrap Versus Parametric Time Series Models.” Water Resources Research 32 (6): 1875–82. https://doi.org/10.1029/96WR00928.

Yatchew, A, and W Hardle. 2006. “Nonparametric State Price Density Estimation Using Constrained Least Squares and the Bootstrap.” Journal of Econometrics 133 (2): 579–99.