We can think of matrices as tensors of order 2. Decomposing matrices is pretty well understood. I know little about decomposing tensors of rank higher than 2. For a flavour of the field, see maybe the tensorly decomposition example notebooks.
tntorch asserts the following are the most popular formats:
- CANDECOMP/PARAFAC (CP) (Kolda and Bader 2009)
- Tucker (De Lathauwer, De Moor, and Vandewalle 2000)
- Tensor Train (TT) (Oseledets 2011)
Applications listed in tntorch:
- Active Subspaces
- ANOVA Decomposition
- Arithmetics
- Automata
- Classification
- Tensor Completion
- Cross-approximation
- Tensor Decompositions
- Exponential Machines
- Boolean Logic
- Polynomial Chaos Expansions
- Sobol Indices
- Vector Fields
References
Anandkumar, Anima, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. 2015. “Tensor Decompositions for Learning Latent Variable Models (A Survey for ALT).” In Algorithmic Learning Theory, edited by Kamalika Chaudhuri, Claudio Gentile, and Sandra Zilles, 19–38. Lecture Notes in Computer Science. Springer International Publishing.
Anandkumar, Animashree, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. 2014. “Tensor Decompositions for Learning Latent Variable Models.” The Journal of Machine Learning Research 15 (1): 2773–2832.
Belkin, Mikhail, Luis Rademacher, and James Voss. 2016. “Basis Learning as an Algorithmic Primitive.” In Journal of Machine Learning Research, 446–87.
Bi, Xuan, Xiwei Tang, Yubai Yuan, Yanqing Zhang, and Annie Qu. 2021. “Tensors in Statistics.” Annual Review of Statistics and Its Application 8 (1): 345–68.
Cui, Tiangang, and Sergey Dolgov. 2022. “Deep Composition of Tensor-Trains Using Squared Inverse Rosenblatt Transports.” Foundations of Computational Mathematics 22 (6): 1863–1922.
De Lathauwer, Lieven, Bart De Moor, and Joos Vandewalle. 2000. “On the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors.” SIAM Journal on Matrix Analysis and Applications 21 (4): 1324–42.
Kolda, Tamara G., and Brett W. Bader. 2009. “Tensor Decompositions and Applications.” SIAM Review 51 (3): 455–500.
Kossaifi, Jean, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, and Anima Anandkumar. 2023. “Multi-Grid Tensorized Fourier Neural Operator for High Resolution PDEs,” February.
Kossaifi, Jean, Yannis Panagakis, Anima Anandkumar, and Maja Pantic. 2019. “TensorLy: Tensor Learning in Python.” Journal of Machine Learning Research 20 (26): 1–6.
Malik, Osman Asif, and Stephen Becker. 2018. “Low-Rank Tucker Decomposition of Large Tensors Using TensorSketch.”
Oseledets, I. V. 2011. “Tensor-Train Decomposition.” SIAM Journal on Scientific Computing 33 (5): 2295–2317.
Pan, Chenjian, Chen Ling, Hongjin He, Liqun Qi, and Yanwei Xu. 2020. “Low-Rank and Sparse Enhanced Tucker Decomposition for Tensor Completion.” arXiv.
Rabanser, Stephan, Oleksandr Shchur, and Stephan Günnemann. 2017. “Introduction to Tensor Decompositions and Their Applications in Machine Learning.”
Rabusseau, Guillaume, and François Denis. 2014. “Learning Negative Mixture Models by Tensor Decompositions.” arXiv:1403.4224 [Cs], March.
Robeva, E. 2016. “Orthogonal Decomposition of Symmetric Tensors.” SIAM Journal on Matrix Analysis and Applications 37 (1): 86–102.
Robeva, Elina, and Anna Seigal. 2016. “Singular Vectors of Orthogonally Decomposable Tensors.” arXiv:1603.09004 [Math], March.
Tenenbaum, J. B., and W. T. Freeman. 2000. “Separating Style and Content with Bilinear Models.” Neural Computation 12 (6): 1247–83.
Tran, Alasdair, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. 2022. “Factorized Fourier Neural Operators.” arXiv.
Zhao, Yiran, and Tiangang Cui. 2023. “Tensor-Based Methods for Sequential State and Parameter Estimation in State Space Models.” arXiv.
No comments yet. Why not leave one?