Notebook on an area about which I know little. Probably mostly notes on contrastive estimation for now?
- Negative sampling and word models on stackexchange
- Lei Mao’s Log Book, Noise Contrastive Estimation
Chehab, Omar, Alexandre Gramfort, and Aapo Hyvarinen. 2022. “The Optimal Noise in Noise-Contrastive Learning Is Not What You Think.” arXiv:2203.01110 [Cs, Stat], March.
Gutmann, Michael U., and Aapo Hyvärinen. 2012. “Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics.” Journal of Machine Learning Research 13 (11): 307–61.
Gutmann, Michael, and Aapo Hyvarinen. n.d. “Noise-Contrastive Estimation: A New Estimation Principle for Unnormalized Statistical Models,” 8.
Le-Khac, Phuc H., Graham Healy, and Alan F. Smeaton. 2020. “Contrastive Representation Learning: A Framework and Review.” IEEE Access 8: 193907–34.
Ma, Zhuang, and Michael Collins. 2018. “Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency.” arXiv:1809.01812 [Cs, Stat], September.
Saunshi, Nikunj, Jordan Ash, Surbhi Goel, Dipendra Misra, Cyril Zhang, Sanjeev Arora, Sham Kakade, and Akshay Krishnamurthy. 2022. “Understanding Contrastive Learning Requires Incorporating Inductive Biases.” arXiv:2202.14037 [Cs], February.
Smith, Noah A., and Jason Eisner. 2005. “Contrastive Estimation: Training Log-Linear Models on Unlabeled Data.” In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL ’05, 354–62. Ann Arbor, Michigan: Association for Computational Linguistics.