Precision matrix estimation

Especially Gaussain

Estimating the inverse of the covariance matrix, the precision matrices.

Two big data problems problems can arise here: large \(p\) (ambient dimension) and large \(n\) (sample size). Large \(p\) is a problem because the covariance matrix is a \(p \times p\) matrixand frequently we need to invert it to calculate some target estimand.

The obvious way

Estimate the covariance matrix then invert it. This is the baseline. πŸ—



πŸ— Wishart priors?



Iterative approximation

Saad (2003)

Conjugate gradients


Bound distance from Covariance

covariance matrix or


Aragam, Bryon, Jiaying Gu, and Qing Zhou. 2017. β€œLearning Large-Scale Bayesian Networks with the Sparsebn Package.” arXiv:1703.04025 [Cs, Stat], March.
Avagyan, Vahe, and Xiaoling Mei. 2022. β€œPrecision Matrix Estimation Under Data Contamination with an Application to Minimum Variance Portfolio Selection.” Communications in Statistics - Simulation and Computation 51 (4): 1381–1400.
Chen, Xiaohui, Mengyu Xu, and Wei Biao Wu. 2013. β€œCovariance and Precision Matrix Estimation for High-Dimensional Time Series.” The Annals of Statistics 41 (6).
Fan, Jianqing, Yuan Liao, and Han Liu. 2016. β€œAn Overview of the Estimation of Large Covariance and Precision Matrices.” The Econometrics Journal 19 (1): C1–32.
Hsieh, Cho-Jui, MΓ‘tyΓ‘s A. Sustik, Inderjit S. Dhillon, and Pradeep D. Ravikumar. 2014. β€œQUIC: Quadratic Approximation for Sparse Inverse Covariance Estimation.” Journal of Machine Learning Research 15 (1): 2911–47.
Hsieh, Cho-Jui, MΓ‘tyΓ‘s A. Sustik, Inderjit S. Dhillon, Pradeep Ravikumar, and Russell A. Poldrack. 2013. β€œBIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables.” In Advances in Neural Information Processing Systems, 16. NIPS’13. Red Hook, NY, USA: Curran Associates Inc.
JankovΓ‘, Jana, and Sara van de Geer. 2015. β€œHonest Confidence Regions and Optimality in High-Dimensional Precision Matrix Estimation.” arXiv:1507.02061 [Math, Stat], July.
Khoshgnauz, Ehsan. 2012. β€œLearning Markov Network Structure Using Brownian Distance Covariance.” arXiv:1206.6361 [Cs, Stat], June.
Kuismin, Markku O., and Mikko J. SillanpÀÀ. 2017. β€œEstimation of Covariance and Precision Matrix, Network Structure, and a View Toward Systems Biology.” WIREs Computational Statistics 9 (6): e1415.
Lam, Clifford, and Jianqing Fan. 2009. β€œSparsistency and Rates of Convergence in Large Covariance Matrix Estimation.” Annals of Statistics 37 (6B): 4254–78.
Le, Thien-Minh, and Ping-Shou Zhong. 2021. β€œHigh-Dimensional Precision Matrix Estimation with a Known Graphical Structure.” arXiv.
Meier, Alexander, Claudia Kirch, and Renate Meyer. 2020. β€œBayesian Nonparametric Analysis of Multivariate Time Series: A Matrix Gamma Process Approach.” Journal of Multivariate Analysis 175 (January): 104560.
Mercer, A. McD. 2000. β€œBounds for A–G, A–H, G–H, and a Family of Inequalities of Ky Fan’s Type, Using a General Method.” Journal of Mathematical Analysis and Applications 243 (1): 163–73.
Moscone, Francesco, Elisa Tosetti, and Veronica Vinciotti. 2017. β€œSparse Estimation of Huge Networks with a Block-Wise Structure.” The Econometrics Journal 20 (3): S61–85.
Pleiss, Geoff, Jacob R. Gardner, Kilian Q. Weinberger, and Andrew Gordon Wilson. 2018. β€œConstant-Time Predictive Distributions for Gaussian Processes.” In. arXiv.
Pourahmadi, Mohsen. 2011. β€œCovariance Estimation: The GLM and Regularization Perspectives.” Statistical Science 26 (3): 369–87.
Saad, Yousef. 2003. Iterative Methods for Sparse Linear Systems: Second Edition. 2nd ed. SIAM.
Sharma, Rajesh. 2008. β€œSome More Inequalities for Arithmetic Mean, Harmonic Mean and Variance.” Journal of Mathematical Inequalities, no. 1: 109–14.
Ubaru, Shashanka, Jie Chen, and Yousef Saad. 2017. β€œFast Estimation of \(tr(f(A))\) via Stochastic Lanczos Quadrature.” SIAM Journal on Matrix Analysis and Applications 38 (4): 1075–99.
Wang, Lingxiao, Xiang Ren, and Quanquan Gu. n.d. β€œPrecision Matrix Estimation in High Dimensional Gaussian Graphical Models with Faster Rates,” 9.
Wu, Wei Biao, and Mohsen Pourahmadi. 2003. β€œNonparametric Estimation of Large Covariance Matrices of Longitudinal Data.” Biometrika 90 (4): 831–44.
Yuan, Ming. 2010. β€œHigh Dimensional Inverse Covariance Matrix Estimation via Linear Programming.” The Journal of Machine Learning Research 11: 26.
Zhang, T., and H. Zou. 2014. β€œSparse Precision Matrix Estimation via Lasso Penalized D-Trace Loss.” Biometrika 101 (1): 103–20.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.