Notebook on an area about which I know little. Probably mostly notes on contrastive learning for now?
Balestriero, Randall, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, et al. 2023. “A Cookbook of Self-Supervised Learning.” arXiv.
Chehab, Omar, Alexandre Gramfort, and Aapo Hyvarinen. 2022. “The Optimal Noise in Noise-Contrastive Learning Is Not What You Think.” arXiv:2203.01110 [Cs, Stat], March.
Gutmann, Michael U., and Aapo Hyvärinen. 2012. “Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics.” Journal of Machine Learning Research 13 (11): 307–61.
Gutmann, Michael, and Aapo Hyvarinen. n.d. “Noise-Contrastive Estimation: A New Estimation Principle for Unnormalized Statistical Models,” 8.
Le-Khac, Phuc H., Graham Healy, and Alan F. Smeaton. 2020. “Contrastive Representation Learning: A Framework and Review.” IEEE Access 8: 193907–34.
Ma, Zhuang, and Michael Collins. 2018. “Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency.” arXiv:1809.01812 [Cs, Stat], September.
Saunshi, Nikunj, Jordan Ash, Surbhi Goel, Dipendra Misra, Cyril Zhang, Sanjeev Arora, Sham Kakade, and Akshay Krishnamurthy. 2022. “Understanding Contrastive Learning Requires Incorporating Inductive Biases.” arXiv:2202.14037 [Cs], February.
Smith, Noah A., and Jason Eisner. 2005. “Contrastive Estimation: Training Log-Linear Models on Unlabeled Data.” In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL ’05, 354–62. Ann Arbor, Michigan: Association for Computational Linguistics.