Precision matrix estimation

Especially Gaussain

November 17, 2014 β€” October 4, 2022

Figure 1

Estimating the inverse of the covariance matrix, the precision matrices.

Two big data problems problems can arise here: large \(p\) (ambient dimension) and large \(n\) (sample size). Large \(p\) is a problem because the covariance matrix is a \(p \times p\) matrixand frequently we need to invert it to calculate some target estimand.

1 The obvious way

Estimate the covariance matrix then invert it. This is the baseline. πŸ—


3 Bayesian

πŸ— Wishart priors?

4 Penalized

5 Structured

6 Iterative approximation

Saad (2003)

6.1 Conjugate gradients

6.2 Lanczos

7 References

Aragam, Gu, and Zhou. 2017. β€œLearning Large-Scale Bayesian Networks with the Sparsebn Package.” arXiv:1703.04025 [Cs, Stat].
Avagyan, and Mei. 2022. β€œPrecision Matrix Estimation Under Data Contamination with an Application to Minimum Variance Portfolio Selection.” Communications in Statistics - Simulation and Computation.
Chen, Xu, and Wu. 2013. β€œCovariance and Precision Matrix Estimation for High-Dimensional Time Series.” The Annals of Statistics.
Fan, Liao, and Liu. 2016. β€œAn Overview of the Estimation of Large Covariance and Precision Matrices.” The Econometrics Journal.
Hsieh, Sustik, Dhillon, et al. 2013. β€œBIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables.” In Advances in Neural Information Processing Systems. NIPS’13.
Hsieh, Sustik, Dhillon, et al. 2014. β€œQUIC: Quadratic Approximation for Sparse Inverse Covariance Estimation.” Journal of Machine Learning Research.
JankovΓ‘, and van de Geer. 2015. β€œHonest Confidence Regions and Optimality in High-Dimensional Precision Matrix Estimation.” arXiv:1507.02061 [Math, Stat].
Khoshgnauz. 2012. β€œLearning Markov Network Structure Using Brownian Distance Covariance.” arXiv:1206.6361 [Cs, Stat].
Kuismin, and SillanpÀÀ. 2017. β€œEstimation of Covariance and Precision Matrix, Network Structure, and a View Toward Systems Biology.” WIREs Computational Statistics.
Lam, and Fan. 2009. β€œSparsistency and Rates of Convergence in Large Covariance Matrix Estimation.” Annals of Statistics.
Le, and Zhong. 2021. β€œHigh-Dimensional Precision Matrix Estimation with a Known Graphical Structure.”
Meier, Kirch, and Meyer. 2020. β€œBayesian Nonparametric Analysis of Multivariate Time Series: A Matrix Gamma Process Approach.” Journal of Multivariate Analysis.
Mercer. 2000. β€œBounds for A–G, A–H, G–H, and a Family of Inequalities of Ky Fan’s Type, Using a General Method.” Journal of Mathematical Analysis and Applications.
Moscone, Tosetti, and Vinciotti. 2017. β€œSparse Estimation of Huge Networks with a Block-Wise Structure.” The Econometrics Journal.
Pleiss, Gardner, Weinberger, et al. 2018. β€œConstant-Time Predictive Distributions for Gaussian Processes.” In.
Pourahmadi. 2011. β€œCovariance Estimation: The GLM and Regularization Perspectives.” Statistical Science.
Saad. 2003. Iterative Methods for Sparse Linear Systems: Second Edition.
Sharma. 2008. β€œSome More Inequalities for Arithmetic Mean, Harmonic Mean and Variance.” Journal of Mathematical Inequalities.
Ubaru, Chen, and Saad. 2017. β€œFast Estimation of \(tr(f(A))\) via Stochastic Lanczos Quadrature.” SIAM Journal on Matrix Analysis and Applications.
Wang, Ren, and Gu. n.d. β€œPrecision Matrix Estimation in High Dimensional Gaussian Graphical Models with Faster Rates.”
Wu, and Pourahmadi. 2003. β€œNonparametric Estimation of Large Covariance Matrices of Longitudinal Data.” Biometrika.
Yuan. 2010. β€œHigh Dimensional Inverse Covariance Matrix Estimation via Linear Programming.” The Journal of Machine Learning Research.
Zhang, and Zou. 2014. β€œSparse Precision Matrix Estimation via Lasso Penalized D-Trace Loss.” Biometrika.