Multiple testing



How to go data mining for models without β€œdredging” for models. (accidentally or otherwise) If you keep on testing models until you find some that fit (which you usually will) how do you know that the fit is in some sense interesting? How sharp will your conclusions be? How does it work when you are testing against a possibly uncountable continuum of hypotheses? (One perspective on sparsity penalties is precisely this, I am told.)

Model selection is this writ small - when you are testing how many variables to include in your model.

In modern high-dimensional models, where you have potentially many explanatory variables, the question of how to handle the combinatorial explosion of possible variables to include, this can also be considered a multiple testing problem. We tend to regard this as a smoothing and model selection problem though.

This all gets more complicated when you think about many people testing many hypothesese in many different experiments then you are going to run into many more issues than just these - also publication bias and suchlike.

Suggestive connection:

Moritz Hardt, The machine learning leaderboard problem:

In this post, I will describe a method to climb the public leaderboard without even looking at the data. The algorithm is so simple and natural that an unwitting analyst might just run it. We will see that in Kaggle’s famous Heritage Health Prize competition this might have propelled a participant from rank around 150 into the top 10 on the public leaderboard without making progress on the actual problem. […]

I get super excited. I keep climbing the leaderboard! Who would’ve thought that this machine learning thing was so easy? So, I go write a blog post on Medium about Big Data and score a job at DeepCompeting.ly, the latest data science startup in the city. Life is pretty sweet. I pick up indoor rock climbing, sign up for wood working classes; I read Proust and books about espresso. Two months later the competition closes and Kaggle releases the final score. What an embarrassment! Wacky boosting did nothing whatsoever on the final test set. I get fired from DeepCompeting.ly days before the buyout. My spouse dumps me. The lease expires. I get evicted from my apartment in the Mission. Inevitably, I hike the Pacific Crest Trail and write a novel about it.

See (Blum and Hardt 2015; Dwork et al. 2015b) for more of that.

P-value hacking

False discovery rate

FDR control…

Familywise error rate

Ε idΓ‘k correction, Bonferroni correction…

Post selection inference

See post selection inference.

References

Abramovich, Felix, Yoav Benjamini, David L. Donoho, and Iain M. Johnstone. 2006. β€œAdapting to Unknown Sparsity by Controlling the False Discovery Rate.” The Annals of Statistics 34 (2): 584–653.
Aickin, M, and H Gensler. 1996. β€œAdjusting for Multiple Testing When Reporting Research Results: The Bonferroni Vs Holm Methods.” American Journal of Public Health 86 (5): 726–28.
Ansley, Craig F., and Robert Kohn. 1985. β€œEstimation, Filtering, and Smoothing in State Space Models with Incompletely Specified Initial Conditions.” The Annals of Statistics 13 (4): 1286–316.
Arnold, Taylor B., and John W. Emerson. 2011. β€œNonparametric Goodness-of-Fit Tests for Discrete Null Distributions.” The R Journal 3 (2): 34–39.
Bach, Francis. 2009. β€œModel-Consistent Sparse Estimation Through the Bootstrap.” arXiv:0901.3202 [Cs, Stat].
Barber, Rina Foygel, and Emmanuel J. CandΓ¨s. 2015. β€œControlling the False Discovery Rate via Knockoffs.” The Annals of Statistics 43 (5): 2055–85.
Bashtannyk, David M., and Rob J. Hyndman. 2001. β€œBandwidth Selection for Kernel Conditional Density Estimation.” Computational Statistics & Data Analysis 36 (3): 279–98.
Bassily, Raef, Kobbi Nissim, Adam Smith, Thomas Steinke, Uri Stemmer, and Jonathan Ullman. 2015. β€œAlgorithmic Stability for Adaptive Data Analysis.” arXiv:1511.02513 [Cs], November.
Benjamini, Yoav. 2010. β€œSimultaneous and Selective Inference: Current Successes and Future Challenges.” Biometrical Journal 52 (6): 708–21.
Benjamini, Yoav, and Yulia Gavrilov. 2009. β€œA Simple Forward Selection Procedure Based on False Discovery Rate Control.” The Annals of Applied Statistics 3 (1): 179–98.
Benjamini, Yoav, and Yosef Hochberg. 1995. β€œControlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing.” Journal of the Royal Statistical Society: Series B (Methodological) 57 (1): 289–300.
Benjamini, Yoav, and Daniel Yekutieli. 2005. β€œFalse Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters.” Journal of the American Statistical Association 100 (469): 71–81.
Berk, Richard, Lawrence Brown, Andreas Buja, Kai Zhang, and Linda Zhao. 2013. β€œValid Post-Selection Inference.” The Annals of Statistics 41 (2): 802–37.
Blum, Avrim, and Moritz Hardt. 2015. β€œThe Ladder: A Reliable Leaderboard for Machine Learning Competitions.” arXiv:1502.04585 [Cs], February.
Buckland, S. T., K. P. Burnham, and N. H. Augustin. 1997. β€œModel Selection: An Integral Part of Inference.” Biometrics 53 (2): 603–18.
BΓΌhlmann, Peter, and Sara van de Geer. 2015. β€œHigh-Dimensional Inference in Misspecified Linear Models.” arXiv:1503.06426 [Stat] 9 (1): 1449–73.
Bunea, Florentina. 2004. β€œConsistent Covariate Selection and Post Model Selection Inference in Semiparametric Regression.” The Annals of Statistics 32 (3): 898–927.
Burnham, Kenneth P., and David R. Anderson. 2004. β€œMultimodel Inference Understanding AIC and BIC in Model Selection.” Sociological Methods & Research 33 (2): 261–304.
Cai, T. Tony, and Wenguang Sun. 2017. β€œLarge-Scale Global and Simultaneous Inference: Estimation and Testing in Very High Dimensions.” Annual Review of Economics 9 (1): 411–39.
CandΓ¨s, Emmanuel J., Yingying Fan, Lucas Janson, and Jinchi Lv. 2016. β€œPanning for Gold: Model-Free Knockoffs for High-Dimensional Controlled Variable Selection.” arXiv Preprint arXiv:1610.02351.
CandΓ¨s, Emmanuel J., J. Romberg, and T. Tao. 2006. β€œRobust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information.” IEEE Transactions on Information Theory 52 (2): 489–509.
CandΓ¨s, Emmanuel J., Michael B. Wakin, and Stephen P. Boyd. 2008. β€œEnhancing Sparsity by Reweighted β„“ 1 Minimization.” Journal of Fourier Analysis and Applications 14 (5-6): 877–905.
Cavanaugh, Joseph E. 1997. β€œUnifying the Derivations for the Akaike and Corrected Akaike Information Criteria.” Statistics & Probability Letters 33 (2): 201–8.
Cavanaugh, Joseph E., and Robert H. Shumway. 1998. β€œAn Akaike Information Criterion for Model Selection in the Presence of Incomplete Data.” Journal of Statistical Planning and Inference 67 (1): 45–65.
Chernozhukov, Victor, Christian Hansen, and Martin Spindler. 2015. β€œValid Post-Selection and Post-Regularization Inference: An Elementary, General Approach.” Annual Review of Economics 7 (1): 649–88.
Chung, Moo K. 2020. β€œIntroduction to Random Fields.” arXiv:2007.09660 [Math, Stat], July.
Claeskens, Gerda, Tatyana Krivobokova, and Jean D. Opsomer. 2009. β€œAsymptotic Properties of Penalized Spline Estimators.” Biometrika 96 (3): 529–44.
Clevenson, M. Lawrence, and James V. Zidek. 1975. β€œSimultaneous Estimation of the Means of Independent Poisson Laws.” Journal of the American Statistical Association 70 (351a): 698–705.
Collings, Bruce J., and Barry H. Margolin. 1985. β€œTesting Goodness of Fit for the Poisson Assumption When Observations Are Not Identically Distributed.” Journal of the American Statistical Association 80 (390): 411–18.
Cox, Christopher R., and Timothy T. Rogers. 2021. β€œFinding Distributed Needles in Neural Haystacks.” Journal of Neuroscience 41 (5): 1019–32.
Cox, D. R., and H. S. Battey. 2017. β€œLarge Numbers of Explanatory Variables, a Semi-Descriptive Analysis.” Proceedings of the National Academy of Sciences 114 (32): 8592–95.
Cule, Erika, Paolo Vineis, and Maria De Iorio. 2011. β€œSignificance Testing in Ridge Regression for Genetic Data.” BMC Bioinformatics 12 (September): 372.
Dai, Ran, and Rina Foygel Barber. 2016. β€œThe Knockoff Filter for FDR Control in Group-Sparse and Multitask Regression.” arXiv Preprint arXiv:1602.03589.
DasGupta, Anirban. 2008. Asymptotic Theory of Statistics and Probability. Springer Texts in Statistics. New York: Springer New York.
Delaigle, Aurore, Peter Hall, and Alexander Meister. 2008. β€œOn Deconvolution with Repeated Measurements.” The Annals of Statistics 36 (2): 665–85.
Dezeure, Ruben, Peter BΓΌhlmann, Lukas Meier, and Nicolai Meinshausen. 2014. β€œHigh-Dimensional Inference: Confidence Intervals, p-Values and R-Software Hdi.” arXiv:1408.4026 [Stat], August.
Donoho, David L., and Iain M. Johnstone. 1995. β€œAdapting to Unknown Smoothness via Wavelet Shrinkage.” Journal of the American Statistical Association 90 (432): 1200–1224.
Donoho, David L., Iain M. Johnstone, Gerard Kerkyacharian, and Dominique Picard. 1995. β€œWavelet Shrinkage: Asymptopia?” Journal of the Royal Statistical Society. Series B (Methodological) 57 (2): 301–69.
Dwork, Cynthia, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Aaron Roth. 2015a. β€œThe Reusable Holdout: Preserving Validity in Adaptive Data Analysis.” Science 349 (6248): 636–38.
Dwork, Cynthia, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Aaron Leon Roth. 2015b. β€œPreserving Statistical Validity in Adaptive Data Analysis.” In Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing - STOC ’15, 117–26. Portland, Oregon, USA: ACM Press.
Efird, Jimmy Thomas, and Susan Searles Nielsen. 2008. β€œA Method to Compute Multiplicity Corrected Confidence Intervals for Odds Ratios and Other Relative Effect Estimates.” International Journal of Environmental Research and Public Health 5 (5): 394–98.
Efron, B. 1979. β€œBootstrap Methods: Another Look at the Jackknife.” The Annals of Statistics 7 (1): 1–26.
Efron, Bradley. 1986. β€œHow Biased Is the Apparent Error Rate of a Prediction Rule?” Journal of the American Statistical Association 81 (394): 461–70.
β€”β€”β€”. 2004a. β€œSelection and Estimation for Large-Scale Simultaneous Inference.”
β€”β€”β€”. 2004b. β€œThe Estimation of Prediction Error.” Journal of the American Statistical Association 99 (467): 619–32.
β€”β€”β€”. 2007. β€œDoing Thousands of Hypothesis Tests at the Same Time.” Metron - International Journal of Statistics LXV (1): 3–21.
β€”β€”β€”. 2008. β€œSimultaneous Inference: When Should Hypothesis Testing Problems Be Combined?” The Annals of Applied Statistics 2 (1): 197–223.
β€”β€”β€”. 2009. β€œEmpirical Bayes Estimates for Large-Scale Prediction Problems.” Journal of the American Statistical Association 104 (487): 1015–28.
β€”β€”β€”. 2010a. β€œThe Future of Indirect Evidence.” Statistical Science 25 (2): 145–57.
β€”β€”β€”. 2010b. β€œCorrelated z-Values and the Accuracy of Large-Scale Statistical Estimates.” Journal of the American Statistical Association 105 (491): 1042–55.
β€”β€”β€”. 2013. Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction. Reprint edition. Cambridge: Cambridge University Press.
Evans, Robin J., and Vanessa Didelez. n.d. β€œRecovering from Selection Bias Using Marginal Structure in Discrete Models.”
Ewald, Karl, and Ulrike Schneider. 2015. β€œConfidence Sets Based on the Lasso Estimator.” arXiv:1507.05315 [Math, Stat], July.
Fan, Jianqing, and Runze Li. 2001. β€œVariable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties.” Journal of the American Statistical Association 96 (456): 1348–60.
Fan, Jianqing, and Jinchi Lv. 2010. β€œA Selective Overview of Variable Selection in High Dimensional Feature Space.” Statistica Sinica 20 (1): 101–48.
Franz, Volker H., and Ulrike von Luxburg. 2014. β€œUnconscious Lie Detection as an Example of a Widespread Fallacy in the Neurosciences.” arXiv:1407.4240 [q-Bio, Stat], July.
Friedman, Jerome, Trevor Hastie, and Rob Tibshirani. 2010. β€œRegularization Paths for Generalized Linear Models via Coordinate Descent.” Journal of Statistical Software 33 (1): 1–22.
Garreau, Damien, RΓ©mi Lajugie, Sylvain Arlot, and Francis Bach. 2014. β€œMetric Learning for Temporal Sequence Alignment.” In Advances in Neural Information Processing Systems 27, edited by Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, 1817–25. Curran Associates, Inc.
Geer, Sara van de, Peter BΓΌhlmann, Ya’acov Ritov, and Ruben Dezeure. 2014. β€œOn Asymptotically Optimal Confidence Regions and Tests for High-Dimensional Models.” The Annals of Statistics 42 (3): 1166–1202.
Geer, Sara van de, and Johannes Lederer. 2011. β€œThe Lasso, Correlated Design, and Improved Oracle Inequalities.” arXiv:1107.0189 [Stat], July.
Gelman, Andrew, and Eric Loken. 2014. β€œThe Statistical Crisis in Science.” American Scientist 102 (6): 460.
Genovese, Christopher, and Larry Wasserman. 2008. β€œAdaptive Confidence Bands.” The Annals of Statistics 36 (2): 875–905.
GonΓ§alves, SΓ­lvia, and Halbert White. 2004. β€œMaximum Likelihood and the Bootstrap for Nonlinear Dynamic Models.” Journal of Econometrics 119 (1): 199–219.
Hardt, Moritz, and Jonathan Ullman. 2014. β€œPreventing False Discovery in Interactive Data Analysis Is Hard.” In Proceedings of the 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, 454–63. FOCS ’14. Washington, DC, USA: IEEE Computer Society.
Hesterberg, Tim, Nam Hee Choi, Lukas Meier, and Chris Fraley. 2008. β€œLeast Angle and β„“1 Penalized Regression: A Review.” Statistics Surveys 2: 61–93.
Hjort, N. L., and M. C. Jones. 1996. β€œLocally Parametric Nonparametric Density Estimation.” The Annals of Statistics 24 (4): 1619–47.
Hjort, Nils Lid. 1992. β€œOn Inference in Parametric Survival Data Models.” International Statistical Review / Revue Internationale de Statistique 60 (3): 355–87.
Hjort, Nils Lid, Mike West, and Sue Leurgans. 1992. β€œSemiparametric Estimation Of Parametric Hazard Rates.” In Survival Analysis: State of the Art, edited by John P. Klein and Prem K. Goel, 211–36. Nato Science 211. Springer Netherlands.
Hurvich, Clifford M., and Chih-Ling Tsai. 1989. β€œRegression and Time Series Model Selection in Small Samples.” Biometrika 76 (2): 297–307.
Ichimura, Hidehiko. 1993. β€œSemiparametric Least Squares (SLS) and Weighted SLS Estimation of Single-Index Models.” Journal of Econometrics 58 (1–2): 71–120.
Ioannidis, John P. 2005. β€œWhy Most Published Research Findings Are False.” PLoS Medicine 2 (8): –124.
Iyengar, Satish, and Joel B. Greenhouse. 1988. β€œSelection Models and the File Drawer Problem.” Statistical Science 3 (1): 109–17.
Jamieson, Kevin, and Lalit Jain. n.d. β€œA Bandit Approach to Multiple Testing with False Discovery Control,” 11.
JankovΓ‘, Jana, and Sara van de Geer. 2015. β€œHonest Confidence Regions and Optimality in High-Dimensional Precision Matrix Estimation.” arXiv:1507.02061 [Math, Stat], July.
Janson, Lucas, William Fithian, and Trevor J. Hastie. 2015. β€œEffective Degrees of Freedom: A Flawed Metaphor.” Biometrika 102 (2): 479–85.
Kaufman, S., and S. Rosset. 2014. β€œWhen Does More Regularization Imply Fewer Degrees of Freedom? Sufficient Conditions and Counterexamples.” Biometrika 101 (4): 771–84.
Konishi, Sadanori, and Genshiro Kitagawa. 1996. β€œGeneralised Information Criteria in Model Selection.” Biometrika 83 (4): 875–90.
Korattikara, Anoop, Yutian Chen, and Max Welling. 2015. β€œSequential Tests for Large-Scale Learning.” Neural Computation 28 (1): 45–70.
Korthauer, Keegan, Patrick K. Kimes, Claire Duvallet, Alejandro Reyes, Ayshwarya Subramanian, Mingxiang Teng, Chinmay Shukla, Eric J. Alm, and Stephanie C. Hicks. 2019. β€œA Practical Guide to Methods Controlling False Discoveries in Computational Biology.” Genome Biology 20 (1): 118.
KΓΌnsch, Hans Rudolf. 1986. β€œDiscrimination Between Monotonic Trends and Long-Range Dependence.” Journal of Applied Probability 23 (4): 1025–30.
Lancichinetti, Andrea, M. Irmak Sirer, Jane X. Wang, Daniel Acuna, Konrad KΓΆrding, and LuΓ­s A. Nunes Amaral. 2015. β€œHigh-Reproducibility and High-Accuracy Method for Automated Topic Classification.” Physical Review X 5 (1): 011007.
Lavergne, Pascal, Samuel Maistre, and Valentin Patilea. 2015. β€œA Significance Test for Covariates in Nonparametric Regression.” Electronic Journal of Statistics 9: 643–78.
Lazzeroni, L C, and A Ray. 2012. β€œThe Cost of Large Numbers of Hypothesis Tests on Power, Effect Size and Sample Size.” Molecular Psychiatry 17 (1): 108–14.
Lee, Jason D., Dennis L. Sun, Yuekai Sun, and Jonathan E. Taylor. 2013. β€œExact Post-Selection Inference, with Application to the Lasso.” arXiv:1311.6238 [Math, Stat], November.
Li, Runze, and Hua Liang. 2008. β€œVariable Selection in Semiparametric Regression Modeling.” The Annals of Statistics 36 (1): 261–86.
Lockhart, Richard, Jonathan Taylor, Ryan J. Tibshirani, and Robert Tibshirani. 2014. β€œA Significance Test for the Lasso.” The Annals of Statistics 42 (2): 413–68.
Meinshausen, Nicolai. 2006. β€œFalse Discovery Control for Multiple Tests of Association Under General Dependence.” Scandinavian Journal of Statistics 33 (2): 227–37.
β€”β€”β€”. 2007. β€œRelaxed Lasso.” Computational Statistics & Data Analysis 52 (1): 374–93.
β€”β€”β€”. 2014. β€œGroup Bound: Confidence Intervals for Groups of Variables in Sparse High Dimensional Regression Without Assumptions on the Design.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), November, n/a–.
Meinshausen, Nicolai, and Peter BΓΌhlmann. 2005. β€œLower Bounds for the Number of False Null Hypotheses for Multiple Testing of Associations Under General Dependence Structures.” Biometrika 92 (4): 893–907.
β€”β€”β€”. 2006. β€œHigh-Dimensional Graphs and Variable Selection with the Lasso.” The Annals of Statistics 34 (3): 1436–62.
β€”β€”β€”. 2010. β€œStability Selection.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 72 (4): 417–73.
Meinshausen, Nicolai, Lukas Meier, and Peter BΓΌhlmann. 2009. β€œP-Values for High-Dimensional Regression.” Journal of the American Statistical Association 104 (488): 1671–81.
Meinshausen, Nicolai, and John Rice. 2006. β€œEstimating the Proportion of False Null Hypotheses Among a Large Number of Independently Tested Hypotheses.” The Annals of Statistics 34 (1): 373–93.
Meinshausen, Nicolai, and Bin Yu. 2009. β€œLasso-Type Recovery of Sparse Representations for High-Dimensional Data.” The Annals of Statistics 37 (1): 246–70.
MΓΌller, Andreas C., and Sven Behnke. 2014. β€œPystruct - Learning Structured Prediction in Python.” Journal of Machine Learning Research 15: 2055–60.
Nickl, Richard, and Sara van de Geer. 2013. β€œConfidence Sets in Sparse Regression.” The Annals of Statistics 41 (6): 2852–76.
Noble, William Stafford. 2009. β€œHow Does Multiple Testing Correction Work?” Nature Biotechnology 27 (12): 1135–37.
Ramsey, Joseph, Madelyn Glymour, Ruben Sanchez-Romero, and Clark Glymour. 2017. β€œA Million Variables and More: The Fast Greedy Equivalence Search Algorithm for Learning High-Dimensional Graphical Causal Models, with an Application to Functional Magnetic Resonance Images.” International Journal of Data Science and Analytics 3 (2): 121–29.
Rosset, Saharon, and Ji Zhu. 2007. β€œPiecewise Linear Regularized Solution Paths.” The Annals of Statistics 35 (3): 1012–30.
Rothman, K. J. 1990. β€œNo adjustments are needed for multiple comparisons.” Epidemiology (Cambridge, Mass.) 1 (1): 43–46.
Rzhetsky, Andrey, Jacob G. Foster, Ian T. Foster, and James A. Evans. 2015. β€œChoosing Experiments to Accelerate Collective Discovery.” Proceedings of the National Academy of Sciences 112 (47): 14569–74.
Siegmund, David O., and Jian Li. 2014. β€œHigher Criticism: P-Values and Criticism.” arXiv:1411.1437 [Math, Stat], November.
Stone, M. 1977. β€œAn Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike’s Criterion.” Journal of the Royal Statistical Society. Series B (Methodological) 39 (1): 44–47.
Storey, John D. 2002. β€œA Direct Approach to False Discovery Rates.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 64 (3): 479–98.
Su, Weijie, Malgorzata Bogdan, and Emmanuel J. CandΓ¨s. 2015. β€œFalse Discoveries Occur Early on the Lasso Path.” arXiv:1511.01957 [Cs, Math, Stat], November.
Taddy, Matt. 2013. β€œOne-Step Estimator Paths for Concave Regularization.” arXiv:1308.5623 [Stat], August.
Tansey, Wesley, Oluwasanmi Koyejo, Russell A. Poldrack, and James G. Scott. 2014. β€œFalse Discovery Rate Smoothing.” arXiv:1411.6144 [Stat], November.
Tansey, Wesley, Oscar Hernan Madrid Padilla, Arun Sai Suggala, and Pradeep Ravikumar. 2015. β€œVector-Space Markov Random Fields via Exponential Families.” In Journal of Machine Learning Research, 684–92.
Taylor, Jonathan, Richard Lockhart, Ryan J. Tibshirani, and Robert Tibshirani. 2014. β€œExact Post-Selection Inference for Forward Stepwise and Least Angle Regression.” arXiv:1401.3889 [Stat], January.
Tibshirani, Ryan J. 2014. β€œA General Framework for Fast Stagewise Algorithms.” arXiv:1408.5801 [Stat], August.
Tibshirani, Ryan J., Alessandro Rinaldo, Robert Tibshirani, and Larry Wasserman. 2015. β€œUniform Asymptotic Inference and the Bootstrap After Model Selection.” arXiv:1506.06266 [Math, Stat], June.
Wasserman, Larry, and Kathryn Roeder. 2009. β€œHigh-Dimensional Variable Selection.” Annals of Statistics 37 (5A): 2178–2201.
Zhang, Cun-Hui, and Stephanie S. Zhang. 2014. β€œConfidence Intervals for Low Dimensional Parameters in High Dimensional Linear Models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 76 (1): 217–42.
Zou, Hui, Trevor Hastie, and Robert Tibshirani. 2007. β€œOn the β€˜Degrees of Freedom’ of the Lasso.” The Annals of Statistics 35 (5): 2173–92.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.