Tensor decompositions

We can think of matrices as tensors of order 2. Decomposing matrices is pretty well understood. I know little about decomposing tensors of rank higher than 2. For a flavour of the field, see maybe the tensorly decomposition example notebooks.

tntorch asserts the following are the most popular formats:

Applications listed in tntorch:


Anandkumar, Anima, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. 2015. β€œTensor Decompositions for Learning Latent Variable Models (A Survey for ALT).” In Algorithmic Learning Theory, edited by Kamalika Chaudhuri, Claudio Gentile, and Sandra Zilles, 19–38. Lecture Notes in Computer Science. Springer International Publishing.
Anandkumar, Animashree, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. 2014. β€œTensor Decompositions for Learning Latent Variable Models.” The Journal of Machine Learning Research 15 (1): 2773–2832.
Belkin, Mikhail, Luis Rademacher, and James Voss. 2016. β€œBasis Learning as an Algorithmic Primitive.” In Journal of Machine Learning Research, 446–87.
Bi, Xuan, Xiwei Tang, Yubai Yuan, Yanqing Zhang, and Annie Qu. 2021. β€œTensors in Statistics.” Annual Review of Statistics and Its Application 8 (1): 345–68.
Cui, Tiangang, and Sergey Dolgov. 2022. β€œDeep Composition of Tensor-Trains Using Squared Inverse Rosenblatt Transports.” Foundations of Computational Mathematics 22 (6): 1863–1922.
De Lathauwer, Lieven, Bart De Moor, and Joos Vandewalle. 2000. β€œOn the Best Rank-1 and Rank-(R1 ,R2 ,. . .,RN) Approximation of Higher-Order Tensors.” SIAM Journal on Matrix Analysis and Applications 21 (4): 1324–42.
Kolda, Tamara G., and Brett W. Bader. 2009. β€œTensor Decompositions and Applications.” SIAM Review 51 (3): 455–500.
Kossaifi, Jean, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, and Anima Anandkumar. 2023. β€œMulti-Grid Tensorized Fourier Neural Operator for High Resolution PDEs,” February.
Kossaifi, Jean, Yannis Panagakis, Anima Anandkumar, and Maja Pantic. 2019. β€œTensorLy: Tensor Learning in Python.” Journal of Machine Learning Research 20 (26): 1–6.
Malik, Osman Asif, and Stephen Becker. 2018. β€œLow-Rank Tucker Decomposition of Large Tensors Using TensorSketch.”
Oseledets, I. V. 2011. β€œTensor-Train Decomposition.” SIAM Journal on Scientific Computing 33 (5): 2295–2317.
Pan, Chenjian, Chen Ling, Hongjin He, Liqun Qi, and Yanwei Xu. 2020. β€œLow-Rank and Sparse Enhanced Tucker Decomposition for Tensor Completion.” arXiv.
Rabanser, Stephan, Oleksandr Shchur, and Stephan GΓΌnnemann. 2017. β€œIntroduction to Tensor Decompositions and Their Applications in Machine Learning.”
Rabusseau, Guillaume, and FranΓ§ois Denis. 2014. β€œLearning Negative Mixture Models by Tensor Decompositions.” arXiv:1403.4224 [Cs], March.
Robeva, E. 2016. β€œOrthogonal Decomposition of Symmetric Tensors.” SIAM Journal on Matrix Analysis and Applications 37 (1): 86–102.
Robeva, Elina, and Anna Seigal. 2016. β€œSingular Vectors of Orthogonally Decomposable Tensors.” arXiv:1603.09004 [Math], March.
Tenenbaum, J. B., and W. T. Freeman. 2000. β€œSeparating Style and Content with Bilinear Models.” Neural Computation 12 (6): 1247–83.
Tran, Alasdair, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. 2022. β€œFactorized Fourier Neural Operators.” arXiv.
Zhao, Yiran, and Tiangang Cui. 2023. β€œTensor-Based Methods for Sequential State and Parameter Estimation in State Space Models.” arXiv.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.