# Non-negative matrix factorisation

October 14, 2019 — October 26, 2023

An interesting special matrix factorisation, where the goal is to decode an *element-wise non-negative* matrix into a product of two smaller matrices, which looks a lot like adaptive sparse coding if you squint at it.

David Austin’s simple intro to Non-negative matrix factorization for the American Mathematical Society is a good inroad to the classic problem.

## 1 Loss functions

TBD

## 2 Optimisation matters

TBD.

## 3 Connection to optimal transport

Optimal transport turns out to have a connection to NNMF (Huynh, Zhao, and Phung 2020; Qian et al. 2016; Scetbon, Cuturi, and Peyré 2021; Schmitz et al. 2018; S. Y. Zhang 2021; Zhao et al. 2020).

## 4 Tools

NMF (R) 🏗

Matlab: Chih-Jen Lin’s nmf.m — “This tool solves NMF by alternative non-negative least squares using projected gradients. It converges faster than the popular multiplicative update approach.”

In this repository, we offer both MPI and OPENMP implementation for MU, HALS and ANLS/BPP based NMF algorithms. This can run off the shelf as well easy to integrate in other source code. These are very highly tuned NMF algorithms to work on super computers. We have tested this software in NERSC as well OLCF cluster. The openmp implementation is tested on many different Linux variants with intel processors. The library works well for both sparse and dense matrix. (Fairbanks et al. 2015; Kannan, Ballard, and Park 2016; Kannan 2016)

## 5 References

*Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion*. AM’18.

*Proceedings of the AAAI Conference on Artificial Intelligence*.

*arXiv:1212.4777 [Cs, Stat]*.

*Positive Transfer Operators and Decay of Correlations*. Advanced Series in Nonlinear Dynamics, v. 16.

*Nonnegative Matrices in the Mathematical Sciences*. Classics in Applied Mathematics 1.0.

*Computational Statistics & Data Analysis*.

*IEEE Transactions on Audio, Speech, and Language Processing*.

*3rd International Symposium on Communications, Control and Signal Processing, 2008. ISCCSP 2008*.

*Proceedings of the 20th International Conference on Digital Audio Effects*.

*Proceedings of the 20th International Joint Conference on Artifical Intelligence*. IJCAI’07.

*IEEE Journal of Selected Topics in Signal Processing*.

*Computational Intelligence and Neuroscience*.

*2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings*.

*arXiv:1405.4275 [Stat]*.

*PLoS Comput Biol*.

*Proceedings of the 2005 SIAM International Conference on Data Mining*. Proceedings.

*IEEE Transactions on Pattern Analysis and Machine Intelligence*.

*Computational Statistics & Data Analysis*.

*Nonnegative Matrices, Positive Operators, and Applications*.

*Proceedings of ISMIR*.

*Journal of Machine Learning Research*.

*Bioinformatics*.

*2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541)*.

*Parallel Computing*, Graph analysis for scientific discovery,.

*Neural Computation*.

*Neural Computation*.

*Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*. KDD ’11.

*IEEE Transactions on Signal Processing*.

*IEEE Transactions on Neural Networks and Learning Systems*.

*International Conference on Machine Learning*.

*Journal of Machine Learning Research*.

*Proceedings of the 2002 12th IEEE Workshop on Neural Networks for Signal Processing, 2002*.

*Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*. KDD ’11.

*2014 48th Asilomar Conference on Signals, Systems and Computers*.

*Advances in Neural Information Processing Systems*.

*Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming*. PPoPP ’16.

*SIAM Journal on Matrix Analysis and Applications*.

*Computer*.

*Nature*.

*Advances in Neural Information Processing Systems 13*.

*Proceedings of the National Academy of Sciences*.

*Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001*.

*Advances in Neural Information Processing Systems 29*.

*Neural Computation*.

*IEEE Transactions on Neural Networks and Learning Systems*.

*arXiv:1607.01649 [Math]*.

*arXiv:1610.08928 [Stat]*.

*Environmetrics*.

*Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence*. IJCAI’16.

*SIAM J. Matrix Anal. Appl.*

*Proceedings of the National Academy of Sciences*.

*Proceedings of the 25th International Conference on Machine Learning*. ICML ’08.

*Transactions on Machine Learning Research*.

*Proceedings of the 38th International Conference on Machine Learning*.

*2007 IEEE Workshop on Machine Learning for Signal Processing*.

*SIAM Journal on Imaging Sciences*.

*Advances in Neural Information Processing Systems 18*.

*IEEE Transactions on Signal Processing*.

*arXiv:1507.03194 [Cs, Stat]*.

*IEEE Transactions on Signal Processing*.

*2008 IEEE International Conference on Acoustics, Speech and Signal Processing*.

*IEEE Transactions on Audio, Speech, and Language Processing*.

*2008 IEEE International Conference on Acoustics, Speech and Signal Processing*.

*In Proc. Asian Conf. On Comp. Vision*.

*SDM*.

*IEEE Transactions on Knowledge and Data Engineering*.

*Machine Audition: Principles, Algorithms and Systems*.

*Algorithmica*.

*Applied and Computational Harmonic Analysis*.

*IEEE International Conference of Data Mining*.

*Knowledge and Information Systems*.

*Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1 - Volume 01*. ICCV ’05.

*Seventh IEEE International Conference on Data Mining, 2007. ICDM 2007*.