Canonical correlation

August 23, 2014 — May 17, 2023

algebra

Bayes

convolution

functional analysis

Hilbert space

linear algebra

machine learning

model selection

nonparametric

statistics

uncertainty

William Press, Canonical Correlation Clarified by Singular Value Decomposition

Most statistical tests are canonical correlation analysis, apparently. (Knapp 1978).

tl;dr classic statistical tests are linear models where your goal is to decide if a coefficient should be regarded as non-zero or not. Jonas Kristoffer Lindeløv explains this perspective: Common statistical tests are linear models. FWIW I found that perspective to be a real 💡 moment.

1 References

Allen-Zhu, and Li. 2017. “Doubly Accelerated Methods for Faster CCA and Generalized Eigendecomposition.” In PMLR.

Bach, and Jordan. 2002. “Kernel Independent Component Analysis.” Journal of Machine Learning Research.

Borello. 2013. “Standardization and Singular Value Decomposition in Canonical Correlation Analysis.”

Cherry. 1996. “Singular Value Decomposition Analysis and Canonical Correlation Analysis.” Journal of Climate.

Cichocki, Lee, Oseledets, et al. 2016. “Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1.” arXiv:1609.00893 [Cs].

Ewerbring, and Luk. 1989. “Canonical Correlations and Generalized SVD: Applications and New Algorithms.” Journal of Computational and Applied Mathematics, Special Issue on Parallel Algorithms for Numerical Linear Algebra,.

Horváth, and Kokoszka. 2012. Inference for functional data with applications. Springer series in statistics.

Knapp. 1978. “Canonical Correlation Analysis: A General Parametric Significance-Testing System.” Psychological Bulletin.

Lopez-Paz, Sra, Smola, et al. 2014. “Randomized Nonlinear Component Analysis.” arXiv:1402.0119 [Cs, Stat].

McWilliams, Balduzzi, and Buhmann. 2013. “Correlated Random Features for Fast Semi-Supervised Learning.” In Advances in Neural Information Processing Systems 26.

Ramsay, and Silverman. 2005. Functional Data Analysis. Springer Series in Statistics.

Witten, Daniela M, and Tibshirani. 2009. “Extensions of Sparse Canonical Correlation Analysis with Applications to Genomic Data.” Statistical Applications in Genetics and Molecular Biology.

Witten, Daniela M., Tibshirani, and Hastie. 2009. “A Penalized Matrix Decomposition, with Applications to Sparse Principal Components and Canonical Correlation Analysis.” Biostatistics.

Yang, and Pan. 2015. “Independence Test for High Dimensional Data Based on Regularized Canonical Correlation Coefficients.” The Annals of Statistics.