# M-estimation

July 11, 2016 — February 17, 2022

Loosely, estimating a quantity by choosing it to be the extremum of a function, or, if it’s well-behaved enough, a zero of its derivative.

Popular with machine learning, where loss-function based methods are ubiquitous. In statistics we see this famously in maximum likelihood estimation and robust estimation, and least squares loss, for which M-estimation provides a unifying formalism with a convenient large sample asymptotic theory.

🏗 Discuss influence function motivation.

## 1 Implied density functions

Common loss function imply a density considered as a maximum likelihood estimation problem.

I assume they did not invent this idea, but Davison and Ortiz (2019) points out that if you have a least-squares-compatible model, usually it can generalise it to any elliptical density, which includes Huber losses and many robust ones as special cases.

## 2 Robust Loss functions

🏗

### 2.1 Huber loss

### 2.2 Hampel loss

## 3 Fitting

Discuss representation (and implementation) in terms of weight functions for least-squares loss.

## 4 GM-estimators

Mallows, Schweppe etc.

🏗

## 5 References

*Advances In Neural Information Processing Systems*.

*Biometrika*.

*Selected Works of Peter J. Bickel*. Selected Works in Probability and Statistics 13.

*Asymptotic Theory of Statistics and Probability*. Springer Texts in Statistics.

*arXiv:1910.14139 [Cs]*.

*arXiv:1310.7320 [Cs, Math, Stat]*.

*Journal of the American Statistical Association*.

*Robust Statistics: The Approach Based on Influence Functions*.

*The Annals of Mathematical Statistics*.

*arXiv:1411.4342 [Stat]*.

*Energy*.

*Annual Review of Statistics and Its Application*.

*Handbook of Statistics*. Robust Inference.

*The Annals of Statistics*.

*Annals of the Institute of Statistical Mathematics*.

*arXiv:2107.02308 [Cs]*.

*Journal of Statistical Planning and Inference*, Robust Statistics and Data Analysis, Part I,.

*Data Segmentation and Model Selection for Computer Vision*.

*Statistics*.

*arXiv:1403.7023 [Math, Stat]*.

*Communications in Statistics - Theory and Methods*.