Precision matrix estimation
Especially Gaussain
2014-11-16 — 2022-10-03
Wherein the Inversion of the Covariance Is Treated as a Task Beset by Large P and Large N, and Iterative Schemes Such as Conjugate Gradients, Lanczos and QUIC Are Presented as Practical Routes to Approximate Precision Matrices
algebra
functional analysis
Hilbert space
kernel tricks
metrics
nonparametric
regression
sparser than thou
statistics
Estimating the inverse of the covariance matrix means estimating the precision matrices.
Two big data problems can arise here: large \(p\) (ambient dimension) and large \(n\) (sample size). Large \(p\) is a problem because the covariance matrix is a \(p \times p\) matrix and we often need to invert it to calculate some target estimand.
1 The obvious way
Estimate the covariance matrix then invert it. This is the baseline. 🚧TODO🚧
2 QUIC
3 Bayesian
🚧TODO🚧 Wishart priors?
4 Penalised
5 Structured
6 Iterative approximation
Saad (2003)
6.1 Conjugate gradients
6.2 Lanczos
7 References
Aragam, Gu, and Zhou. 2017. “Learning Large-Scale Bayesian Networks with the Sparsebn Package.” arXiv:1703.04025 [Cs, Stat].
Avagyan, and Mei. 2022. “Precision Matrix Estimation Under Data Contamination with an Application to Minimum Variance Portfolio Selection.” Communications in Statistics - Simulation and Computation.
Chen, Xu, and Wu. 2013. “Covariance and Precision Matrix Estimation for High-Dimensional Time Series.” The Annals of Statistics.
Fan, Liao, and Liu. 2016. “An Overview of the Estimation of Large Covariance and Precision Matrices.” The Econometrics Journal.
Hsieh, Sustik, Dhillon, et al. 2013. “BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables.” In Advances in Neural Information Processing Systems. NIPS’13.
Hsieh, Sustik, Dhillon, et al. 2014. “QUIC: Quadratic Approximation for Sparse Inverse Covariance Estimation.” Journal of Machine Learning Research.
Janková, and van de Geer. 2015. “Honest Confidence Regions and Optimality in High-Dimensional Precision Matrix Estimation.” arXiv:1507.02061 [Math, Stat].
Khoshgnauz. 2012. “Learning Markov Network Structure Using Brownian Distance Covariance.” arXiv:1206.6361 [Cs, Stat].
Kuismin, and Sillanpää. 2017. “Estimation of Covariance and Precision Matrix, Network Structure, and a View Toward Systems Biology.” WIREs Computational Statistics.
Lam, and Fan. 2009. “Sparsistency and Rates of Convergence in Large Covariance Matrix Estimation.” Annals of Statistics.
Le, and Zhong. 2021. “High-Dimensional Precision Matrix Estimation with a Known Graphical Structure.”
Meier, Kirch, and Meyer. 2020. “Bayesian Nonparametric Analysis of Multivariate Time Series: A Matrix Gamma Process Approach.” Journal of Multivariate Analysis.
Mercer. 2000. “Bounds for A–G, A–H, G–H, and a Family of Inequalities of Ky Fan’s Type, Using a General Method.” Journal of Mathematical Analysis and Applications.
Moscone, Tosetti, and Vinciotti. 2017. “Sparse Estimation of Huge Networks with a Block-Wise Structure.” The Econometrics Journal.
Pleiss, Gardner, Weinberger, et al. 2018. “Constant-Time Predictive Distributions for Gaussian Processes.” In.
Pourahmadi. 2011. “Covariance Estimation: The GLM and Regularization Perspectives.” Statistical Science.
Sharma. 2008. “Some More Inequalities for Arithmetic Mean, Harmonic Mean and Variance.” Journal of Mathematical Inequalities.
Ubaru, Chen, and Saad. 2017. “Fast Estimation of \(tr(f(A))\) via Stochastic Lanczos Quadrature.” SIAM Journal on Matrix Analysis and Applications.
Wang, Ren, and Gu. n.d. “Precision Matrix Estimation in High Dimensional Gaussian Graphical Models with Faster Rates.”
Wu, and Pourahmadi. 2003. “Nonparametric Estimation of Large Covariance Matrices of Longitudinal Data.” Biometrika.
Yuan. 2010. “High Dimensional Inverse Covariance Matrix Estimation via Linear Programming.” The Journal of Machine Learning Research.
Zhang, and Zou. 2014. “Sparse Precision Matrix Estimation via Lasso Penalized D-Trace Loss.” Biometrika.