Loosely, estimating a quantity by choosing it to be the extremum of a function, or, if itβs well-behaved enough, a zero of its derivative.

Popular with machine learning, where loss-function based methods are ubiquitous. In statistics we see this famously in maximum likelihood estimation and robust estimation, and least squares loss, for which M-estimation provides a unifying formalism with a convenient large sample asymptotic theory.

π Discuss influence function motivation.

## Implied density functions

Common loss function imply a density considered as a maximum_likelihood estimation problem.

## Robust Loss functions

π

### Huber loss

### Hampel loss

## Fitting

Discuss representation (and implementation) in terms of weight functions for least-squares loss.

## GM-estimators

Mallows, Schweppe etc.

π

## References

Advani, Madhu, and Surya Ganguli. 2016. βAn Equivalence Between High Dimensional Bayes Optimal Inference and M-Estimation.β In

*Advances In Neural Information Processing Systems*.
Barndorff-Nielsen, O. 1983. βOn a Formula for the Distribution of the Maximum Likelihood Estimator.β

*Biometrika*70 (2): 343β65.
BΓΌhlmann, Peter. 2014. βRobust Statistics.β In

*Selected Works of Peter J. Bickel*, edited by Jianqing Fan, Yaβacov Ritov, and C. F. Jeff Wu, 51β98. Selected Works in Probability and Statistics 13. Springer New York.
DasGupta, Anirban. 2008.

*Asymptotic Theory of Statistics and Probability*. Springer Texts in Statistics. New York: Springer New York.
Donoho, David L., and Andrea Montanari. 2013. βHigh Dimensional Robust M-Estimation: Asymptotic Variance via Approximate Message Passing.β

*arXiv:1310.7320 [Cs, Math, Stat]*, October.
Geer, Sara van de. 2014. βWorst Possible Sub-Directions in High-Dimensional Models.β In

*arXiv:1403.7023 [Math, Stat]*. Vol. 131.
Hampel, Frank R. 1974. βThe Influence Curve and Its Role in Robust Estimation.β

*Journal of the American Statistical Association*69 (346): 383β93.
Hampel, Frank R., Elvezio M. Ronchetti, Peter J. Rousseeuw, and Werner A. Stahel. 2011.

*Robust Statistics: The Approach Based on Influence Functions*. John Wiley & Sons.
Huber, Peter J. 1964. βRobust Estimation of a Location Parameter.β

*The Annals of Mathematical Statistics*35 (1): 73β101.
Kandasamy, Kirthevasan, Akshay Krishnamurthy, Barnabas Poczos, Larry Wasserman, and James M. Robins. 2014. βInfluence Functions for Machine Learning: Nonparametric Estimators for Entropies, Divergences and Mutual Informations.β

*arXiv:1411.4342 [Stat]*, November.
KΓΌmmel, Reiner. 1982. βThe Impact of Energy on Industrial Growth.β

*Energy*7 (2): 189β203.
Markatou, M., and E. Ronchetti. 1997. β3 Robust Inference: The Approach Based on Influence Functions.β In

*Handbook of Statistics*, edited by BT - Handbook of Statistics, 15:49β75. Robust Inference. Elsevier.
Maronna, Ricardo Antonio. 1976. βRobust M-Estimators of Multivariate Location and Scatter.β

*The Annals of Statistics*4 (1): 51β67.
Mondal, Debashis, and Donald B. Percival. 2010. βM-Estimation of Wavelet Variance.β

*Annals of the Institute of Statistical Mathematics*64 (1): 27β53.
Ronchetti, E. 2000. βRobust Regression Methods and Model Selection.β In

*Data Segmentation and Model Selection for Computer Vision*, edited by Alireza Bab-Hadiashar and David Suter, 31β40. Springer New York.
Ronchetti, Elvezio. 1997. βRobust Inference by Influence Functions.β

*Journal of Statistical Planning and Inference*, Robust Statistics and Data Analysis, Part I, 57 (1): 59β72.
Tharmaratnam, Kukatharmini, and Gerda Claeskens. 2013. βA Comparison of Robust Versions of the AIC Based on M-, S- and MM-Estimators.β

*Statistics*47 (1): 216β35.
## No comments yet. Why not leave one?