Tensor decompositions
August 15, 2016 — February 21, 2023
algebra
Hilbert space
We can think of matrices as tensors of order 2. Decomposing matrices is pretty well understood. I know little about decomposing tensors of rank higher than 2. For a flavour of the field, see maybe the tensorly decomposition example notebooks.
tntorch asserts the following are the most popular formats:
- CANDECOMP/PARAFAC (CP) (Kolda and Bader 2009)
- Tucker (De Lathauwer, De Moor, and Vandewalle 2000)
- Tensor Train (TT) (Oseledets 2011)
Applications listed in tntorch:
- Active Subspaces
- ANOVA Decomposition
- Arithmetics
- Automata
- Classification
- Tensor Completion
- Cross-approximation
- Tensor Decompositions
- Exponential Machines
- Boolean Logic
- Polynomial Chaos Expansions
- Sobol Indices
- Vector Fields
1 Tooling
2 References
Anandkumar, Animashree, Ge, Hsu, et al. 2014. “Tensor Decompositions for Learning Latent Variable Models.” The Journal of Machine Learning Research.
Anandkumar, Anima, Ge, Hsu, et al. 2015. “Tensor Decompositions for Learning Latent Variable Models (A Survey for ALT).” In Algorithmic Learning Theory. Lecture Notes in Computer Science.
Belkin, Rademacher, and Voss. 2016. “Basis Learning as an Algorithmic Primitive.” In Journal of Machine Learning Research.
Bi, Tang, Yuan, et al. 2021. “Tensors in Statistics.” Annual Review of Statistics and Its Application.
Cui, and Dolgov. 2022. “Deep Composition of Tensor-Trains Using Squared Inverse Rosenblatt Transports.” Foundations of Computational Mathematics.
De Lathauwer, De Moor, and Vandewalle. 2000. “On the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors.” SIAM Journal on Matrix Analysis and Applications.
Kolda, and Bader. 2009. “Tensor Decompositions and Applications.” SIAM Review.
Kossaifi, Kovachki, Azizzadenesheli, et al. 2023. “Multi-Grid Tensorized Fourier Neural Operator for High Resolution PDEs.”
Kossaifi, Panagakis, Anandkumar, et al. 2019. “TensorLy: Tensor Learning in Python.” Journal of Machine Learning Research.
Malik, and Becker. 2018. “Low-Rank Tucker Decomposition of Large Tensors Using TensorSketch.”
Oseledets. 2011. “Tensor-Train Decomposition.” SIAM Journal on Scientific Computing.
Pan, Ling, He, et al. 2020. “Low-Rank and Sparse Enhanced Tucker Decomposition for Tensor Completion.”
Rabanser, Shchur, and Günnemann. 2017. “Introduction to Tensor Decompositions and Their Applications in Machine Learning.”
Rabusseau, and Denis. 2014. “Learning Negative Mixture Models by Tensor Decompositions.” arXiv:1403.4224 [Cs].
Robeva, E. 2016. “Orthogonal Decomposition of Symmetric Tensors.” SIAM Journal on Matrix Analysis and Applications.
Robeva, Elina, and Seigal. 2016. “Singular Vectors of Orthogonally Decomposable Tensors.” arXiv:1603.09004 [Math].
Tenenbaum, and Freeman. 2000. “Separating Style and Content with Bilinear Models.” Neural Computation.
Tran, Mathews, Xie, et al. 2022. “Factorized Fourier Neural Operators.”
Zhao, and Cui. 2023. “Tensor-Based Methods for Sequential State and Parameter Estimation in State Space Models.”