# Kernel density estimators

A nonparametric method of approximating something from data by assuming that it’s close to the data distribution convolved with some kernel.

This is especially popular the target is a probability density function; Then you are working with a kernel density estimator.

• “Effective local sample size”

• understand the Frechet derivative + Wiener filtering construction used to derive the optimal kernel shape in .

### Bandwidth/kernel selection in density estimation

Bernacchia and Pigolotti (2011) has a neat hack: “self consistency” for simultaneous kernel and distribution inference, i.e. simultaneous deconvolution and bandwidth selection. The idea is removing bias by using simple spectral methods, thereby estimating a kernel which in a certain sense would generate the data that you just observed. The results look similar to finite-sample corrections for Gaussian scale parameter estimates, but are not quite Gaussian.

Question: could it work with mixture models too?

### Mixture models

Where the number of kernels does not grow as fast as the number of data points, this becomes a mixture model; Or, if you’d like, kernel density estimates are a limiting case of mixture model estimates.

They are so clearly similar that I think it best we not make them both feel awkward by dithering about where the free parameters are. Anyway, they are filed separately. discuss some useful things common to various convex combination estimators.

### Does this work with uncertain point locations?

The fact we can write the kernel density estimate as an integral with a convolution of Dirac deltas immediately suggests that we could write it as a convolution of something else, such as Gaussians. Can we recover well-behaved estimates in that case? This would be a kind of hierarchical model, possibly a typical Bayesian one.

### Does this work with asymmetric kernels?

Almost all the kernel estimates I’ve seen require KDEs to be symmetric, because of Cline’s argument that asymmetric kernels are inadmissible in the class of all (possibly multivariate) densities. Presumably this implies $$\mathcal{C}_1$$ distributions, i.e. once-differentiable ones. In particular admissible kernels are those which have “nonnegative Fourier transforms bounded by 1”, which implies symmetry about the axis. If we have an a priori constrained class of densities, this might not apply.

### Fast Gauss Transform and Fast multipole methods

How to make these methods computationally feasible at scale. See Fast Gauss Transform and other related fast multipole methods.

## References

Aalen, Odd O. 1978. The Annals of Statistics 6 (4): 701–26.
Adelfio, Giada, and Frederic Paik Schoenberg. 2009. Annals of the Institute of Statistical Mathematics 61 (4): 929–48.
Baddeley, Adrian, and Rolf Turner. 2006. In Case Studies in Spatial Point Process Modeling, edited by Adrian Baddeley, Pablo Gregori, Jorge Mateu, Radu Stoica, and Dietrich Stoyan, 23–74. Lecture Notes in Statistics 185. Springer New York.
Baddeley, A., R. Turner, J. Møller, and M. Hazelton. 2005. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67 (5): 617–66.
Barnes, Josh, and Piet Hut. 1986. Nature 324 (6096): 446–49.
Bashtannyk, David M., and Rob J. Hyndman. 2001. Computational Statistics & Data Analysis 36 (3): 279–98.
Battey, Heather, and Han Liu. 2013. arXiv:1308.3968 [Stat], August.
Berman, Mark, and Peter Diggle. 1989. Journal of the Royal Statistical Society. Series B (Methodological) 51 (1): 81–92.
Bernacchia, Alberto, and Simone Pigolotti. 2011. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 (3): 407–22.
Botev, Z. I., J. F. Grotowski, and D. P. Kroese. 2010. The Annals of Statistics 38 (5): 2916–57.
Cline, Daren B. H. 1988. The Annals of Statistics 16 (4): 1421–27.
Crisan, Dan, and Joaquín Míguez. 2014. Bernoulli 20 (4): 1879–929.
Cucala, Lionel. 2008. Scandinavian Journal of Statistics 35 (2): 322–34.
Díaz-Avalos, Carlos, P. Juan, and J. Mateu. 2012. Stochastic Environmental Research and Risk Assessment 27 (5): 1193–1205.
Diggle, Peter. 1985. Journal of the Royal Statistical Society. Series C (Applied Statistics) 34 (2): 138–47.
Diggle, Peter J. 1979. Biometrics 35 (1): 87–101.
Doosti, Hassan, and Peter Hall. 2015. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 78 (2): 445–62.
Ellis, Steven P. 1991. Stochastic Processes and Their Applications 39 (2): 345–58.
Geenens, Gery. 2014. Journal of the American Statistical Association 109 (505): 346–58.
Geer, Sara van de. 1996. Journal of Nonparametric Statistics 6 (4): 293–310.
Greengard, L., and J. Strain. 1991. SIAM Journal on Scientific and Statistical Computing 12 (1): 79–94.
Hall, Peter. 1987. The Annals of Statistics 15 (4): 1491–1519.
Hall, Peter, and Byeong U. Park. 2002. The Annals of Statistics 30 (5): 1460–79.
Helmers, Roelof, I. Wayan Mangku, and Ričardas Zitikis. 2003. Journal of Multivariate Analysis 84 (1): 19–39.
Ho, Nhat, and Stephen G. Walker. 2020. arXiv:2012.14482 [Math, Stat], December.
Ibragimov, I. 2001. In Institute of Mathematical Statistics Lecture Notes - Monograph Series, 359–83. Beachwood, OH: Institute of Mathematical Statistics.
Jones, M.C., and D.A. Henderson. 2009. Computational Statistics & Data Analysis 53 (10): 3726–33.
Koenker, Roger, and Ivan Mizera. 2006. Advances in Statistical Modeling and Inference, 613–34.
Lieshout, Marie-Colette N. M. van. 2011. Methodology and Computing in Applied Probability 14 (3): 567–78.
Liu, Guangcan, Shiyu Chang, and Yi Ma. 2012. arXiv:1209.2082 [Cs], September.
Malec, Peter, and Melanie Schienle. 2014. Computational Statistics & Data Analysis 72 (April): 57–76.
Marshall, Jonathan C., and Martin L. Hazelton. 2010. Journal of Multivariate Analysis 101 (4): 949–63.
O’Brien, Travis A., Karthik Kashinath, Nicholas R. Cavanaugh, William D. Collins, and John P. O’Brien. 2016. Computational Statistics & Data Analysis 101 (September): 148–60.
Panaretos, Victor M., and Kjell Konis. 2012. Journal of the American Statistical Association 107 (499): 1085–95.
Rathbun, Stephen L. 1996. Biometrics, 226–42.
Raykar, Vikas C., and Ramani Duraiswami. 2005.
Silverman, B. W. 1982. The Annals of Statistics 10 (3): 795–810.
Smith, Evan, and Michael S. Lewicki. 2005. Neural Computation 17 (1): 19–45.
Stein, Michael L. 2005. Journal of the American Statistical Association 100 (469): 310–21.
Yang, Changjiang, Ramani Duraiswami, and Larry S. Davis. 2004. In Advances in Neural Information Processing Systems, 1561–68.
Yang, Changjiang, Ramani Duraiswami, Nail A. Gumerov, and Larry Davis. 2003. In Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2, 464–64. ICCV ’03. Washington, DC, USA: IEEE Computer Society.
Zeevi, Assaf J., and Ronny Meir. 1997. Neural Networks: The Official Journal of the International Neural Network Society 10 (1): 99–109.
Zhang, Shunpu, and Rohana J. Karunamuni. 2010. Journal of Nonparametric Statistics 22 (1): 81–104.

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.