# Covariance matrix estimation

## Esp Gaussian Estimating the thing that is given to you by oracles in statistics homework assignments: the covariance matrix. Or, if the data is indexed by some parameter we might consider the covariance kernel. We are especially interested in this in Gaussian processes, where the covariance structure characterises the process up to its mean.

I am not introducing a complete theory of covariance estimation here, merely mentioning a couple of tidbits for future reference.

Two big data problems problems can arise here: large $$p$$ (ambient dimension) and large $$n$$ (sample size). Large $$p$$ is a problem because the covariance matrix is a $$p \times p$$ matrixand frequently we need to invert it to calculate some target estimand.

Often life can be made not too bad for large $$n$$ with Gaussian structure because, essentially, the problem has a nice exponential family structure and hence has sufficient statistics.

## Bayesian

Inverse Wishart priors. 🏗 Other?

## Precision estimation

The workhorse of learning graphical models under linearity and Gaussianity. See precision estimation for a more complete treatment.

## Continuous

See kernel learning.

## Parametric

### on a lattice

Estimating a stationary covariance function on a regular lattice? That is a whole field of its own. Useful keywords include circulant embedding. Although strictly more general than Gaussian processes on a lattice, it is often used in that context and some extra results are on that page for now.

## Unordered

Thanks to Rothman (2010) I now think about covariance estimates as being different in ordered versus exchangeable data.

## Sandwich estimators For robust covariances of vector data. AKA Heteroskedasticity-consistent covariance estimators. Incorporating Eicker-Huber-White sandwich estimator, Andrews kernel HAC estimator, Newey-West and others. For an intro see Achim Zeileis, Open-Source Econometric Computing in R.

## Incoming

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.