Kernel density estimators
2016-03-05 — 2016-08-18
Wherein kernel density estimators are presented as data distributions convolved with a kernel, and bandwidth is selected by a self-consistency spectral method that yields effective local sample sizes and corrections.
A nonparametric method of approximating something from data by assuming that it’s close to the data distribution convolved with some kernel.
This is especially popular when the target is a probability density function; then you are working with a kernel density estimator.
To learn about:
“Effective local sample size”
Understand the Fréchet derivative + Wiener filtering construction used to derive the optimal kernel shape in (Bernacchia and Pigolotti 2011; O’Brien et al. 2016).
0.1 Bandwidth/kernel selection in density estimation
Bernacchia and Pigolotti (2011) has a neat hack: “self consistency” for simultaneous kernel and distribution inference, i.e. simultaneous deconvolution and bandwidth selection. The idea is removing bias by using simple spectral methods, thereby estimating a kernel which in a certain sense would generate the data that you just observed. The results look similar to finite-sample corrections for Gaussian scale parameter estimates but are not quite Gaussian.
Question: could it work with mixture models too?
0.2 Mixture models
Where the number of kernels does not grow as fast as the number of data points, this becomes a mixture model; or, if you’d like, kernel density estimates are a limiting case of mixture model estimates.
They are so clearly similar that I think it best we not make them both feel awkward by dithering about where the free parameters are. Anyway, they are filed separately. (Battey and Liu 2013; van de Geer 1996; Zeevi and Meir 1997) discuss some useful things common to various convex combination estimators.
0.3 Does this work with uncertain point locations?
The fact we can write the kernel density estimate as an integral with a convolution of Dirac deltas immediately suggests that we could write it as a convolution of something else, such as Gaussians. Can we recover well-behaved estimates in that case? This would be a kind of hierarchical model, possibly a typical Bayesian one.
0.4 Does this work with asymmetric kernels?
Almost all the kernel estimates I’ve seen require KDEs to be symmetric, because of Cline’s argument that asymmetric kernels are inadmissible in the class of all (possibly multivariate) densities. Presumably this implies \(\mathcal{C}_1\) distributions, i.e. once-differentiable ones. In particular, admissible kernels are those which have “nonnegative Fourier transforms bounded by 1,” which implies symmetry about the axis. If we have an a priori constrained class of densities, this might not apply.
0.5 Fast Gauss Transform and Fast multipole methods
How to make these methods computationally feasible at scale. See Fast Gauss Transform and other related fast multipole methods.