Precision matrix estimation

Especially Gaussain

November 17, 2014 — October 4, 2022

algebra
functional analysis
Hilbert space
kernel tricks
metrics
nonparametric
regression
sparser than thou
statistics
Figure 1

Estimating the inverse of the covariance matrix, the precision matrices.

Two big data problems can arise here: large \(p\) (ambient dimension) and large \(n\) (sample size). Large \(p\) is a problem because the covariance matrix is a \(p \times p\) matrix and frequently we need to invert it to calculate some target estimand.

1 The obvious way

Estimate the covariance matrix then invert it. This is the baseline. 🏗

2 QUIC

3 Bayesian

🏗 Wishart priors?

4 Penalized

5 Structured

6 Iterative approximation

Saad (2003)

6.1 Conjugate gradients

6.2 Lanczos

7 References

Aragam, Gu, and Zhou. 2017. “Learning Large-Scale Bayesian Networks with the Sparsebn Package.” arXiv:1703.04025 [Cs, Stat].
Avagyan, and Mei. 2022. “Precision Matrix Estimation Under Data Contamination with an Application to Minimum Variance Portfolio Selection.” Communications in Statistics - Simulation and Computation.
Chen, Xu, and Wu. 2013. “Covariance and Precision Matrix Estimation for High-Dimensional Time Series.” The Annals of Statistics.
Fan, Liao, and Liu. 2016. “An Overview of the Estimation of Large Covariance and Precision Matrices.” The Econometrics Journal.
Hsieh, Sustik, Dhillon, et al. 2013. “BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables.” In Advances in Neural Information Processing Systems. NIPS’13.
Hsieh, Sustik, Dhillon, et al. 2014. “QUIC: Quadratic Approximation for Sparse Inverse Covariance Estimation.” Journal of Machine Learning Research.
Janková, and van de Geer. 2015. “Honest Confidence Regions and Optimality in High-Dimensional Precision Matrix Estimation.” arXiv:1507.02061 [Math, Stat].
Khoshgnauz. 2012. “Learning Markov Network Structure Using Brownian Distance Covariance.” arXiv:1206.6361 [Cs, Stat].
Kuismin, and Sillanpää. 2017. “Estimation of Covariance and Precision Matrix, Network Structure, and a View Toward Systems Biology.” WIREs Computational Statistics.
Lam, and Fan. 2009. “Sparsistency and Rates of Convergence in Large Covariance Matrix Estimation.” Annals of Statistics.
Le, and Zhong. 2021. “High-Dimensional Precision Matrix Estimation with a Known Graphical Structure.”
Meier, Kirch, and Meyer. 2020. “Bayesian Nonparametric Analysis of Multivariate Time Series: A Matrix Gamma Process Approach.” Journal of Multivariate Analysis.
Mercer. 2000. “Bounds for A–G, A–H, G–H, and a Family of Inequalities of Ky Fan’s Type, Using a General Method.” Journal of Mathematical Analysis and Applications.
Moscone, Tosetti, and Vinciotti. 2017. “Sparse Estimation of Huge Networks with a Block-Wise Structure.” The Econometrics Journal.
Pleiss, Gardner, Weinberger, et al. 2018. “Constant-Time Predictive Distributions for Gaussian Processes.” In.
Pourahmadi. 2011. “Covariance Estimation: The GLM and Regularization Perspectives.” Statistical Science.
Saad. 2003. Iterative Methods for Sparse Linear Systems: Second Edition.
Sharma. 2008. “Some More Inequalities for Arithmetic Mean, Harmonic Mean and Variance.” Journal of Mathematical Inequalities.
Ubaru, Chen, and Saad. 2017. “Fast Estimation of \(tr(f(A))\) via Stochastic Lanczos Quadrature.” SIAM Journal on Matrix Analysis and Applications.
Wang, Ren, and Gu. n.d. “Precision Matrix Estimation in High Dimensional Gaussian Graphical Models with Faster Rates.”
Wu, and Pourahmadi. 2003. “Nonparametric Estimation of Large Covariance Matrices of Longitudinal Data.” Biometrika.
Yuan. 2010. “High Dimensional Inverse Covariance Matrix Estimation via Linear Programming.” The Journal of Machine Learning Research.
Zhang, and Zou. 2014. “Sparse Precision Matrix Estimation via Lasso Penalized D-Trace Loss.” Biometrika.