# Sparse regression

Penalised regression where the penalties are sparsifying. The prediction losses could be anything β likelihood, least-squares, robust Huberised losses, absolute deviation etc.

I will play fast and loose with terminology here regarding theoretical and empirical losses, and the statistical models we attempt to fit.

In nonparametric statistics we might estimate simultaneously what look like many, many parameters, which we constrain in some clever fashion, which usually boils down to something we can interpret as a smoothing parameters, controlling how many factors we still have to consider, from a subset of the original.

I will usually discuss our intent to minimise prediction error, but one could also try to minimise model selection error too.

Then we have a simultaneous estimation and model selection procedure, probably a specific sparse model selection procedure and we possibly have to choose clever optimisation method to do the whole thing fast. Related to compressed sensing, but here we consider sampling complexity and measurement error.

See also matrix factorisations, optimisation, multiple testing, concentration inequalities, sparse flavoured icecream.

π disambiguate the optimisation technologies at play β iteratively reweighted least squares etc.

Now! A set of headings under which I will try to understand some things, mostly the LASSO variants.

## LASSO

Quadratic loss penalty, absolute coefficient penalty. We estimate the regression coefficients $$\beta$$ by solving

\begin{aligned} \hat{\beta} = \underset{\beta \in \mathbb{R}^p}{\text{argmin}} \: \frac{1}{2} \| y - {\bf X} \beta \|_2^2 + \lambda \| \beta \|_1, \end{aligned}

The penalty coefficient $$\lambda$$ is left for you to choose, but one of the magical properties of the lasso is that it is easy to test many possible values of $$\lambda$$ at low marginal cost.

Popular because, amongst other reasons, it turns out to be in practice fast and convenient, and amenable to various performance accelerations e.g. aggressive approximate variable selection.

π This is the one with famous oracle properties if you choose $$\lambda$$ correctly. Hsi Zouβs paper on this (Zou 2006) is readable. I am having trouble digesting Sara van de Geerβs paper on the Generalised Lasso, but it seems to offer me guarantees for something very similar to the Adaptive Lasso, but with far more general assumptions on the model and loss functions, and some finite sample guarnatees.

## LARS

A confusing one; LASSO and LARS are not the same thing but you can use one to calculate the other? Something like that? I need to work this one through with a pencil and paper.

## Graph LASSO

As used in graphical models. π

## Elastic net

Combination of $$L_1$$ and $$L_2$$ penalties. π

## Grouped LASSO

AFAICT this is the usual LASSO but with grouped factors. See .

## Model selection

Can be fiddly with sparse regression, which couples variable selection tightly with parameter estimation. See sparse model selection.

## Debiased LASSO

There exist a few versions, but the one I have needed is , section 2.1. See also and . (π relation to ?)

## Sparse basis expansions

Wavelets etc; mostly handled under sparse dictionary bases.

## Sparse neural nets

That is, sparse regressions as the layers in a neural network? Sure thing.

## Other coefficient penalties

Put a weird penalty on the coefficients! E.g. βSmoothly Clipped Absolute Deviationβ (SCAD). π

## Other prediction losses

Put a weird penalty on the error! MAD prediction penalty, lasso-coefficient penalty, etc.

See for some implementations using e.g. maximum absolute prediction error.

## Implementations

Hastie, Friedman etaβs glmnet for R is fast and well-regarded, and has a MATLAB version. Hereβs how to use it for adaptive lasso. Kenneth Tay has implemented elasticnet penalty for any GLM in glmnet.

SPAMS (C++, MATLAB, R, python) by Mairal, looks interesting. Itβs an optimisation library for many, many sparse problems.

liblinear also include lasso-type solvers, as well as support-vector regression.

## Tidbits

Sparse regression as a universal classifier explainer? Local Interpretable Model-agnostic Explanations uses LASSO for model interpretation this. (See the blog post, or the source.

## References

Abramovich, Felix, Yoav Benjamini, David L. Donoho, and Iain M. Johnstone. 2006. The Annals of Statistics 34 (2): 584β653.
Aghasi, Alireza, Nam Nguyen, and Justin Romberg. 2016. arXiv:1611.05162 [Cs, Stat], November.
Aragam, Bryon, Arash A. Amini, and Qing Zhou. 2015. arXiv:1511.08963 [Cs, Math, Stat], November.
Azadkia, Mona, and Sourav Chatterjee. 2019. arXiv:1910.12327 [Cs, Math, Stat], December.
Azizyan, Martin, Akshay Krishnamurthy, and Aarti Singh. 2015. arXiv:1506.00898 [Cs, Math, Stat], June.
Bach, Francis. 2009. arXiv:0901.3202 [Cs, Stat].
Bach, Francis, Rodolphe Jenatton, and Julien Mairal. 2011. Optimization With Sparsity-Inducing Penalties. Foundations and Trends(r) in Machine Learning 1.0. Now Publishers Inc.
Bahmani, Sohail, and Justin Romberg. 2014. arXiv:1501.00046 [Cs, Math, Stat], December.
Banerjee, Arindam, Sheng Chen, Farideh Fazayeli, and Vidyashankar Sivakumar. 2014. In Advances in Neural Information Processing Systems 27, edited by Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, 1556β64. Curran Associates, Inc.
Banerjee, Onureena, Laurent El Ghaoui, and Alexandre dβAspremont. 2008. Journal of Machine Learning Research 9 (Mar): 485β516.
Barber, Rina Foygel, and Emmanuel J. CandΓ¨s. 2015. The Annals of Statistics 43 (5): 2055β85.
Barbier, Jean. 2015. arXiv:1511.01650 [Cs, Math], November.
Baron, Dror, Shriram Sarvotham, and Richard G. Baraniuk. 2010. IEEE Transactions on Signal Processing 58 (1): 269β80.
Barron, Andrew R., Albert Cohen, Wolfgang Dahmen, and Ronald A. DeVore. 2008. The Annals of Statistics 36 (1): 64β94.
Barron, Andrew R., Cong Huang, Jonathan Q. Li, and Xi Luo. 2008. In Information Theory Workshop, 2008. ITWβ08. IEEE, 247β57. IEEE.
Battiti, Roberto. 1992. Neural Computation 4 (2): 141β66.
Bayati, M., and A. Montanari. 2012. IEEE Transactions on Information Theory 58 (4): 1997β2017.
Bellec, Pierre C., and Alexandre B. Tsybakov. 2016. arXiv:1609.06675 [Math, Stat], September.
Belloni, Alexandre, Victor Chernozhukov, and Lie Wang. 2011. Biometrika 98 (4): 791β806.
Berk, Richard, Lawrence Brown, Andreas Buja, Kai Zhang, and Linda Zhao. 2013. The Annals of Statistics 41 (2): 802β37.
Bertin, K., E. Le Pennec, and V. Rivoirard. 2011. Annales de lβInstitut Henri PoincarΓ©, ProbabilitΓ©s Et Statistiques 47 (1): 43β74.
Bian, Wei, Xiaojun Chen, and Yinyu Ye. 2014. Mathematical Programming 149 (1-2): 301β27.
Bien, Jacob, Irina Gaynanova, Johannes Lederer, and Christian L. MΓΌller. 2018. Journal of Computational and Graphical Statistics 27 (1): 23β33.
Bloniarz, Adam, Hanzhong Liu, Cun-Hui Zhang, Jasjeet Sekhon, and Bin Yu. 2015. arXiv:1507.03652 [Math, Stat], July.
Bondell, Howard D., Arun Krishna, and Sujit K. Ghosh. 2010. Biometrics 66 (4): 1069β77.
Borgs, Christian, Jennifer T. Chayes, Henry Cohn, and Yufei Zhao. 2014. arXiv:1401.2906 [Math], January.
Bottou, LΓ©on, Frank E. Curtis, and Jorge Nocedal. 2016. arXiv:1606.04838 [Cs, Math, Stat], June.
Breiman, Leo. 1995. Technometrics 37 (4): 373β84.
Bruckstein, A. M., Michael Elad, and M. Zibulevsky. 2008. IEEE Transactions on Information Theory 54 (11): 4813β20.
Brunton, Steven L., Joshua L. Proctor, and J. Nathan Kutz. 2016. Proceedings of the National Academy of Sciences 113 (15): 3932β37.
Bu, Yunqi, and Johannes Lederer. 2017. arXiv:1704.02739 [Stat], April.
BΓΌhlmann, Peter, and Sara van de Geer. 2011. In Statistics for High-Dimensional Data, 77β97. Springer Series in Statistics. Springer Berlin Heidelberg.
βββ. 2015. arXiv:1503.06426 [Stat] 9 (1): 1449β73.
Bunea, Florentina, Alexandre B. Tsybakov, and Marten H. Wegkamp. 2007a. In Learning Theory, edited by Nader H. Bshouty and Claudio Gentile, 530β43. Lecture Notes in Computer Science. Springer Berlin Heidelberg.
Bunea, Florentina, Alexandre Tsybakov, and Marten Wegkamp. 2007b. Electronic Journal of Statistics 1: 169β94.
CandΓ¨s, Emmanuel J., and Mark A. Davenport. 2011. arXiv:1104.5246 [Cs, Math, Stat], April.
CandΓ¨s, Emmanuel J., Yingying Fan, Lucas Janson, and Jinchi Lv. 2016. arXiv Preprint arXiv:1610.02351.
CandΓ¨s, Emmanuel J., and Carlos Fernandez-Granda. 2013. Journal of Fourier Analysis and Applications 19 (6): 1229β54.
CandΓ¨s, Emmanuel J., and Y. Plan. 2010. βMatrix Completion With Noise.β Proceedings of the IEEE 98 (6): 925β36.
CandΓ¨s, Emmanuel J., Justin K. Romberg, and Terence Tao. 2006. Communications on Pure and Applied Mathematics 59 (8): 1207β23.
CandΓ¨s, Emmanuel J., Michael B. Wakin, and Stephen P. Boyd. 2008. Journal of Fourier Analysis and Applications 14 (5-6): 877β905.
Carmi, Avishy Y. 2013. Digital Signal Processing 23 (3): 751β70.
βββ. 2014. In Compressed Sensing & Sparse Filtering, edited by Avishy Y. Carmi, Lyudmila Mihaylova, and Simon J. Godsill, 281β324. Signals and Communication Technology. Springer Berlin Heidelberg.
Cevher, Volkan, Marco F. Duarte, Chinmay Hegde, and Richard Baraniuk. 2009. In Advances in Neural Information Processing Systems, 257β64. Curran Associates, Inc.
Chartrand, R., and Wotao Yin. 2008. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008, 3869β72.
Chatterjee, Sourav. 2020. arXiv:1909.10140 [Math, Stat], January.
Chen, Minhua, J. Silva, J. Paisley, Chunping Wang, D. Dunson, and L. Carin. 2010. IEEE Transactions on Signal Processing 58 (12): 6140β55.
Chen, Xiaojun. 2012. Mathematical Programming 134 (1): 71β99.
Chen, Yen-Chi, and Yu-Xiang Wang. n.d.
Chen, Y., and A. O. Hero. 2012. IEEE Transactions on Signal Processing 60 (8): 3978β87.
Chernozhukov, Victor, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. 2016. arXiv:1608.00060 [Econ, Stat], July.
Chernozhukov, Victor, Christian Hansen, Yuan Liao, and Yinchu Zhu. 2018. arXiv:1812.08089 [Math, Stat], December.
Chernozhukov, Victor, Whitney K. Newey, and Rahul Singh. 2018. arXiv:1809.05224 [Econ, Math, Stat], September.
Chetverikov, Denis, Zhipeng Liao, and Victor Chernozhukov. 2016. βOn Cross-Validated Lasso.β arXiv:1605.02214 [Math, Stat], May.
Chichignoud, MichaΓ«l, Johannes Lederer, and Martin Wainwright. 2014. arXiv:1410.0247 [Math, Stat], October.
Dai, Ran, and Rina Foygel Barber. 2016. arXiv Preprint arXiv:1602.03589.
Daneshmand, Hadi, Manuel Gomez-Rodriguez, Le Song, and Bernhard SchΓΆlkopf. 2014. In ICML.
Descloux, Pascaline, and Sylvain Sardy. 2018. arXiv:1805.05133 [Stat], May.
Diaconis, Persi, and David Freedman. 1984. The Annals of Statistics 12 (3): 793β815.
Dossal, Charles, Maher Kachour, Jalal M. Fadili, Gabriel PeyrΓ©, and Christophe Chesneau. 2011. arXiv:1111.1162 [Cs, Math, Stat], November.
Efron, Bradley, Trevor Hastie, Iain Johnstone, and Robert Tibshirani. 2004. βLeast Angle Regression.β The Annals of Statistics 32 (2): 407β99.
El Karoui, Noureddine. 2008. University of California, Berkeley 36 (6): 2717β56.
Elhamifar, E., and R. Vidal. 2013. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (11): 2765β81.
Engebretsen, Solveig, and Jon Bohlin. 2019. Clinical Epigenetics 11 (1): 123.
Ewald, Karl, and Ulrike Schneider. 2015. arXiv:1507.05315 [Math, Stat], July.
Fan, Jianqing, and Runze Li. 2001. Journal of the American Statistical Association 96 (456): 1348β60.
Fan, Jianqing, and Jinchi Lv. 2010. Statistica Sinica 20 (1): 101β48.
Fan, Rong-En, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. βLIBLINEAR: A Library for Large Linear Classification.β Journal of Machine Learning Research 9: 1871β74.
Flynn, Cheryl J., Clifford M. Hurvich, and Jeffrey S. Simonoff. 2013. arXiv:1302.2068 [Stat], February.
Foygel, Rina, and Nathan Srebro. 2011. arXiv:1108.0373 [Math, Stat], August.
Friedman, Jerome, Trevor Hastie, Holger HΓΆfling, and Robert Tibshirani. 2007. The Annals of Applied Statistics 1 (2): 302β32.
Friedman, Jerome, Trevor Hastie, and Robert Tibshirani. 2008. Biostatistics 9 (3): 432β41.
Fu, Fei, and Qing Zhou. 2013. Journal of the American Statistical Association 108 (501): 288β300.
Gasso, G., A. Rakotomamonjy, and S. Canu. 2009. IEEE Transactions on Signal Processing 57 (12): 4686β98.
Geer, Sara A. van de. 2008. The Annals of Statistics 36 (2): 614β45.
Geer, Sara A. van de, Peter BΓΌhlmann, and Shuheng Zhou. 2011. Electronic Journal of Statistics 5: 688β749.
Geer, Sara van de. 2007. βThe Deterministic Lasso.β
βββ. 2014a. Scandinavian Journal of Statistics 41 (1): 72β86.
βββ. 2014b. In arXiv:1403.7023 [Math, Stat]. Vol. 131.
βββ. 2014c. arXiv:1409.8557 [Math, Stat], September.
βββ. 2016. Estimation and Testing Under Sparsity. Vol. 2159. Lecture Notes in Mathematics. Cham: Springer International Publishing.
Geer, Sara van de, Peter BΓΌhlmann, Yaβacov Ritov, and Ruben Dezeure. 2014. The Annals of Statistics 42 (3): 1166β1202.
Ghadimi, Saeed, and Guanghui Lan. 2013a. SIAM Journal on Optimization 23 (4): 2341β68.
βββ. 2013b. arXiv:1310.3787 [Math], October.
Girolami, Mark. 2001. Neural Computation 13 (11): 2517β32.
Giryes, Raja, Guillermo Sapiro, and Alex M. Bronstein. 2014. arXiv:1412.5896 [Cs, Math, Stat], December.
Greenhill, Catherine, Mikhail Isaev, Matthew Kwan, and Brendan D. McKay. 2016. arXiv:1606.01586 [Math], June.
Gu, Jiaying, Fei Fu, and Qing Zhou. 2014. arXiv:1403.2310 [Stat], March.
Gui, Jiang, and Hongzhe Li. 2005. Bioinformatics 21 (13): 3001β8.
Gupta, Pawan, and Marianna Pensky. 2016. arXiv:1605.07913 [Math, Stat], May.
Hall, Peter, Jiashun Jin, and Hugh Miller. 2014. Bernoulli 20 (3): 1647β71.
Hall, Peter, and Jing-Hao Xue. 2014. Computational Statistics & Data Analysis 71 (March): 694β708.
Hallac, David, Jure Leskovec, and Stephen Boyd. 2015. arXiv:1507.00280 [Cs, Math, Stat], July.
Hansen, Niels Richard, Patricia Reynaud-Bouret, and Vincent Rivoirard. 2015. Bernoulli 21 (1): 83β143.
Hastie, Trevor J., Tibshirani, Rob, and Martin J. Wainwright. 2015. Statistical Learning with Sparsity: The Lasso and Generalizations. Boca Raton: Chapman and Hall/CRC.
Hawe, S., M. Kleinsteuber, and K. Diepold. 2013. IEEE Transactions on Image Processing 22 (6): 2138β50.
He, Dan, Irina Rish, and Laxmi Parida. 2014. βTransductive HSIC Lasso.β In Proceedings of the 2014 SIAM International Conference on Data Mining, edited by Mohammed Zaki, Zoran Obradovic, Pang Ning Tan, Arindam Banerjee, Chandrika Kamath, and Srinivasan Parthasarathy, 154β62. Proceedings. Philadelphia, PA: Society for Industrial and Applied Mathematics.
Hebiri, Mohamed, and Sara A. van de Geer. 2011. Electronic Journal of Statistics 5: 1184β1226.
Hegde, Chinmay, and Richard G. Baraniuk. 2012. IEEE Transactions on Information Theory 58 (12): 7204β14.
Hegde, Chinmay, Piotr Indyk, and Ludwig Schmidt. 2015. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), 928β37.
Hesterberg, Tim, Nam Hee Choi, Lukas Meier, and Chris Fraley. 2008. Statistics Surveys 2: 61β93.
Hirose, Kei, Shohei Tateishi, and Sadanori Konishi. 2011. arXiv:1109.2411 [Stat], September.
Hormati, A., O. Roy, Y.M. Lu, and M. Vetterli. 2010. IEEE Transactions on Signal Processing 58 (3): 1095β1109.
Hsieh, Cho-Jui, MΓ‘tyΓ‘s A. Sustik, Inderjit S. Dhillon, and Pradeep D. Ravikumar. 2014. Journal of Machine Learning Research 15 (1): 2911β47.
Hu, Tao, Cengiz Pehlevan, and Dmitri B. Chklovskii. 2014. In 2014 48th Asilomar Conference on Signals, Systems and Computers.
Huang, Cong, G. L. H. Cheang, and Andrew R. Barron. 2008.
Ishwaran, Hemant, and J. Sunil Rao. 2005. The Annals of Statistics 33 (2): 730β73.
JankovΓ‘, Jana, and Sara van de Geer. 2016. arXiv:1610.01353 [Math, Stat], October.
Janson, Lucas, William Fithian, and Trevor J. Hastie. 2015. Biometrika 102 (2): 479β85.
Javanmard, Adel, and Andrea Montanari. 2014. Journal of Machine Learning Research 15 (1): 2869β909.
Jung, Alexander. 2013. In Advances in Neural Information Processing Systems 29.
KabΓ‘n, Ata. 2014. In Journal of Machine Learning Research, 448β56.
Kato, Kengo. 2009. Journal of Multivariate Analysis 100 (7): 1338β52.
Kim, Yongdai, Sunghoon Kwon, and Hosik Choi. 2012. Journal of Machine Learning Research 13 (Apr): 1037β57.
Koltchinskii, Prof Vladimir. 2011. Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Mathematics Γcole dβΓtΓ© de ProbabilitΓ©s de Saint-Flour 2033. Heidelberg: Springer.
Koppel, Alec, Garrett Warnell, Ethan Stump, and Alejandro Ribeiro. 2016. arXiv:1612.04111 [Cs, Stat], December.
Kowalski, Matthieu, and Bruno TorrΓ©sani. 2009. In SPARSβ09-Signal Processing with Adaptive Sparse Structured Representations.
KrΓ€mer, Nicole, Juliane SchΓ€fer, and Anne-Laure Boulesteix. 2009. BMC Bioinformatics 10 (1): 384.
Lam, Clifford, and Jianqing Fan. 2009. Annals of Statistics 37 (6B): 4254β78.
Lambert-Lacroix, Sophie, and Laurent Zwald. 2011. Electronic Journal of Statistics 5: 1015β53.
Langford, John, Lihong Li, and Tong Zhang. 2009. In Advances in Neural Information Processing Systems 21, edited by D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, 905β12. Curran Associates, Inc.
Lederer, Johannes, and Michael Vogt. 2020. arXiv:2004.11554 [Stat], April.
Lee, Jason D., Dennis L. Sun, Yuekai Sun, and Jonathan E. Taylor. 2013. arXiv:1311.6238 [Math, Stat], November.
Lemhadri, Ismael, Feng Ruan, Louis Abraham, and Robert Tibshirani. 2021. Journal of Machine Learning Research 22 (127): 1β29.
Li, Wei, and Johannes Lederer. 2019. Journal of Statistical Planning and Inference 202 (September): 80β98.
Lockhart, Richard, Jonathan Taylor, Ryan J. Tibshirani, and Robert Tibshirani. 2014. The Annals of Statistics 42 (2): 413β68.
Lu, W., Y. Goldberg, and J. P. Fine. 2012. Biometrika 99 (3): 717β31.
Lundberg, Scott M, and Su-In Lee. 2017. In Advances in Neural Information Processing Systems. Vol. 30. Curran Associates, Inc.
Mahoney, Michael W. 2016. arXiv Preprint arXiv:1608.04845.
Mairal, J. 2015. SIAM Journal on Optimization 25 (2): 829β55.
Mazumder, Rahul, Jerome H Friedman, and Trevor J. Hastie. 2009. Stanford University.
Meier, Lukas, Sara van de Geer, and Peter BΓΌhlmann. 2008. Group 70 (Part 1): 53β71.
Meinshausen, Nicolai, and Peter BΓΌhlmann. 2006. The Annals of Statistics 34 (3): 1436β62.
Meinshausen, Nicolai, and Bin Yu. 2009. The Annals of Statistics 37 (1): 246β70.
Molchanov, Dmitry, Arsenii Ashukha, and Dmitry Vetrov. 2017. In Proceedings of ICML.
Montanari, Andrea. 2012. Compressed Sensing: Theory and Applications, 394β438.
Mousavi, Ali, and Richard G. Baraniuk. 2017. In ICASSP.
MΓΌller, Patric, and Sara van de Geer. 2015. TEST, April.
Naik, Prasad A., and Chih-Ling Tsai. 2001. Biometrika 88 (3): 821β32.
Nam, Sangnam, and R. Gribonval. 2012. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 5397β5400.
Needell, D., and J. A. Tropp. 2008. arXiv:0803.2392 [Cs, Math], March.
Nesterov, Yu. 2012. Mathematical Programming 140 (1): 125β61.
Neville, Sarah E., John T. Ormerod, and M. P. Wand. 2014. Electronic Journal of Statistics 8 (1): 1113β51.
Ngiam, Jiquan, Zhenghao Chen, Sonia A. Bhaskar, Pang W. Koh, and Andrew Y. Ng. 2011. βSparse Filtering.β In Advances in Neural Information Processing Systems 24, edited by J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, 1125β33. Curran Associates, Inc.
Nickl, Richard, and Sara van de Geer. 2013. The Annals of Statistics 41 (6): 2852β76.
Oymak, S., A. Jalali, M. Fazel, and B. Hassibi. 2013. In 2013 IEEE 52nd Annual Conference on Decision and Control (CDC), 6019β24.
Peleg, Tomer, Yonina C. Eldar, and Michael Elad. 2010. IEEE Transactions on Signal Processing 60 (5): 2286β2303.
Portnoy, Stephen, and Roger Koenker. 1997. Statistical Science 12 (4): 279β300.
Pouget-Abadie, Jean, and Thibaut Horel. 2015. In Proceedings of The 32nd International Conference on Machine Learning.
Pourahmadi, Mohsen. 2011. Statistical Science 26 (3): 369β87.
Qian, Wei, and Yuhong Yang. 2012. Annals of the Institute of Statistical Mathematics 65 (2): 295β318.
Qin, Zhiwei, Katya Scheinberg, and Donald Goldfarb. 2013. Mathematical Programming Computation 5 (2): 143β69.
Rahimi, Ali, and Benjamin Recht. 2009. In Advances in Neural Information Processing Systems, 1313β20. Curran Associates, Inc.
Ravikumar, Pradeep, Martin J. Wainwright, Garvesh Raskutti, and Bin Yu. 2011. Electronic Journal of Statistics 5: 935β80.
Ravishankar, Saiprasad, and Yoram Bresler. 2015. arXiv:1501.02923 [Cs, Stat], January.
Ravishankar, S., and Y. Bresler. 2015. IEEE Transactions on Signal Processing 63 (9): 2389β2404.
Reynaud-Bouret, Patricia. 2003. Probability Theory and Related Fields 126 (1).
Reynaud-Bouret, Patricia, and Sophie Schbath. 2010. The Annals of Statistics 38 (5): 2781β2822.
Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2016. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135β44. KDD β16. New York, NY, USA: ACM.
Rish, Irina, and Genady Grabarnik. 2014. In Compressed Sensing & Sparse Filtering, edited by Avishy Y. Carmi, Lyudmila Mihaylova, and Simon J. Godsill, 77β93. Signals and Communication Technology. Springer Berlin Heidelberg.
Rish, Irina, and Genady Ya Grabarnik. 2015. Sparse Modeling: Theory, Algorithms, and Applications. Chapman & Hall/CRC Machine Learning & Pattern Recognition Series. Boca Raton, FL: CRC Press, Taylor & Francis Group.
RoΔkovΓ‘, Veronika, and Edward I. George. 2018. βThe Spike-and-Slab LASSO.β Journal of the American Statistical Association 113 (521): 431β44.
Sashank J. Reddi, Suvrit Sra, BarnabΓ‘s PΓ³czΓ³s, and Alex Smola. 1995.
Schelldorfer, JΓΌrg, Peter BΓΌhlmann, and Sara Van De Geer. 2011. Scandinavian Journal of Statistics 38 (2): 197β214.
Semenova, Lesia, Cynthia Rudin, and Ronald Parr. 2021. arXiv:1908.01755 [Cs, Stat], April.
She, Yiyuan, and Art B. Owen. 2010.
Shen, Xiaotong, and Hsin-Cheng Huang. 2006. Journal of the American Statistical Association 101 (474): 554β68.
Shen, Xiaotong, Hsin-Cheng Huang, and Jimmy Ye. 2004. Technometrics 46 (3): 306β17.
Shen, Xiaotong, and Jianming Ye. 2002. βAdaptive Model Selection.β Journal of the American Statistical Association 97 (457): 210β21.
Simon, Noah, Jerome Friedman, Trevor Hastie, and Rob Tibshirani. 2011. Journal of Statistical Software 39 (5).
Smith, Virginia, Simone Forte, Michael I. Jordan, and Martin Jaggi. 2015. arXiv:1512.04011 [Cs], December.
Soh, Yong Sheng, and Venkat Chandrasekaran. 2017. arXiv:1701.01207 [Cs, Math, Stat], January.
Soltani, Mohammadreza, and Chinmay Hegde. 2016. Statistics 7: 9.
Starck, J. L., Michael Elad, and David L. Donoho. 2005. IEEE Transactions on Image Processing 14 (10): 1570β82.
Stine, Robert A. 2004. The Annals of Statistics 32 (2): 407β99.
Su, Weijie, Malgorzata Bogdan, and Emmanuel J. CandΓ¨s. 2015. arXiv:1511.01957 [Cs, Math, Stat], November.
Taddy, Matt. 2013. arXiv:1308.5623 [Stat], August.
Tarr, Garth, Samuel MΓΌller, and Alan H. Welsh. 2018. Journal of Statistical Software 83 (1): 1β28.
Thisted, Ronald A. 1997. Statistical Science 12 (4): 296β98.
Thrampoulidis, Chrtistos, Ehsan Abbasi, and Babak Hassibi. 2015. In Advances in Neural Information Processing Systems 28, edited by C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, R. Garnett, and R. Garnett, 3402β10. Curran Associates, Inc.
Tibshirani, Robert. 1996. Journal of the Royal Statistical Society. Series B (Methodological) 58 (1): 267β88.
βββ. 2011. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 (3): 273β82.
Tibshirani, Ryan J. 2014. arXiv:1408.5801 [Stat], August.
Trofimov, Ilya, and Alexander Genkin. 2015. In Analysis of Images, Social Networks and Texts, edited by Mikhail Yu Khachay, Natalia Konstantinova, Alexander Panchenko, Dmitry I. Ignatov, and Valeri G. Labunets, 243β54. Communications in Computer and Information Science 542. Springer International Publishing.
βββ. 2016. arXiv:1611.02101 [Cs, Stat], November.
Tropp, J. A., and S. J. Wright. 2010. Proceedings of the IEEE 98 (6): 948β58.
Tschannen, Michael, and Helmut BΓΆlcskei. 2016. arXiv:1612.03450 [Cs, Math, Stat], December.
Uematsu, Yoshimasa. 2015. arXiv:1504.06706 [Math, Stat], April.
Unser, Michael A., and Pouya Tafti. 2014. An Introduction to Sparse Stochastic Processes. New York: Cambridge University Press.
Unser, M., P. D. Tafti, A. Amini, and H. Kirshner. 2014. IEEE Transactions on Information Theory 60 (5): 3036β51.
Unser, M., P. D. Tafti, and Q. Sun. 2014. IEEE Transactions on Information Theory 60 (3): 1945β62.
Veitch, Victor, and Daniel M. Roy. 2015. arXiv:1512.03099 [Cs, Math, Stat], December.
Wahba, Grace. 1990. Spline Models for Observational Data. SIAM.
Wang, Hansheng, Guodong Li, and Guohua Jiang. 2007. Journal of Business & Economic Statistics 25 (3): 347β55.
Wang, L., M. D. Gordon, and J. Zhu. 2006. In Sixth International Conference on Data Mining (ICDMβ06), 690β700.
Wang, Zhangyang, Shiyu Chang, Qing Ling, Shuai Huang, Xia Hu, Honghui Shi, and Thomas S. Huang. 2016. In.
Wisdom, Scott, Thomas Powers, James Pitton, and Les Atlas. 2016. In Advances in Neural Information Processing Systems 29.
Woodworth, Joseph, and Rick Chartrand. 2015. arXiv:1504.02923 [Cs, Math], April.
Wright, S. J., R. D. Nowak, and M. A. T. Figueiredo. 2009. IEEE Transactions on Signal Processing 57 (7): 2479β93.
Wu, Tong Tong, and Kenneth Lange. 2008. The Annals of Applied Statistics 2 (1): 224β44.
Xu, H., C. Caramanis, and S. Mannor. 2010. βRobust Regression and Lasso.β IEEE Transactions on Information Theory 56 (7): 3561β74.
βββ. 2012. IEEE Transactions on Pattern Analysis and Machine Intelligence 34 (1): 187β93.
Yaghoobi, M., Sangnam Nam, R. Gribonval, and M.E. Davies. 2012. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 5409β12.
Yang, Wenzhuo, and Huan Xu. 2013. In ICML (3), 585β93.
Yoshida, Ryo, and Mike West. 2010. Journal of Machine Learning Research 11 (May): 1771β98.
Yuan, Ming, and Yi Lin. 2006. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68 (1): 49β67.
βββ. 2007. Biometrika 94 (1): 19β35.
Yun, Sangwoon, and Kim-Chuan Toh. 2009. Computational Optimization and Applications 48 (2): 273β307.
Zhang, Cun-Hui. 2010. The Annals of Statistics 38 (2): 894β942.
Zhang, Cun-Hui, and Stephanie S. Zhang. 2014. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 76 (1): 217β42.
Zhang, Lijun, Tianbao Yang, Rong Jin, and Zhi-Hua Zhou. 2015. arXiv:1511.03766 [Cs], November.
Zhang, Yiyun, Runze Li, and Chih-Ling Tsai. 2010. Journal of the American Statistical Association 105 (489): 312β23.
Zhao, Peng, Guilherme Rocha, and Bin Yu. 2006.
βββ. 2009. The Annals of Statistics 37 (6A): 3468β97.
Zhao, Peng, and Bin Yu. 2006. Journal of Machine Learning Research 7 (Nov): 2541β63.
Zhao, Tuo, Han Liu, and Tong Zhang. 2018. The Annals of Statistics 46 (1): 180β218.
Zhou, Tianyi, Dacheng Tao, and Xindong Wu. 2011. Data Mining and Knowledge Discovery 22 (3): 340β71.
Zou, Hui. 2006. Journal of the American Statistical Association 101 (476): 1418β29.
Zou, Hui, and Trevor Hastie. 2005. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67 (2): 301β20.
Zou, Hui, Trevor Hastie, and Robert Tibshirani. 2007. The Annals of Statistics 35 (5): 2173β92.
Zou, Hui, and Runze Li. 2008. The Annals of Statistics 36 (4): 1509β33.

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.