Recommender systems

2020-11-30 – 2022-08-15
Maybe related 🀷:

Assumed audience:

People who have not heard about recommender systems yet

Not my core area, but I need a landing page to refer to because this is coming up in conversations a lot at the moment.

Users who liked apostasy also enjoyed…

Recommender systems are a weird area. There have been some high profile examples of recommender models going back a long time (e.g. the Netflix Prize) and they are clearly commercially important. At time of writing, though, there is relatively little material introducing these systems to wide audiences, compared to say linear regressions and image classifiers and so on. Possibly this is something about who owns the data to make these tools go, which is to say, there are relatively few open data sets for recommender systems?

I would like to refer to a simple and good introduction to the area, but I do not know one, so what follows a scruffy word salad with some helpful links.

YouTube’s defence of their recommender system introduces in vague outline a lot of the features that make it go on the way to defending them from allegations of feeding extremism [citation needed].

Zoo

Starting from an overview by Javier we can find many approaches to more abstract recommender systems.

  1. Most Popular recommendations (the baseline)
  2. Item-User similarity based recommendations
  3. kNN Collaborative Filtering recommendations
  4. Gradient Boosting Machine based recommendations
  5. Non-Negative Matrix Factorization recommendations
  6. Factorization Machines (Rendle 2010)
  7. Field Aware Factorization Machines (Juan et al. 2016)
  8. Deep Learning based recommendations (Wide and Deep, Cheng et al. (2016))
  9. Neural Collaborative Filtering (He et al. 2017)
  10. Neural Graph Collaborative Filtering (Xiang Wang et al. 2019)
  11. Variational Autoencoders for Collaborative Filtering (Liang et al. 2018)

Not mentioned in that list filtering based on monitoring the brains of subjects (Davis III, SpapΓ©, and Ruotsalo 2021).

A different take is in Microsoft’s Best Practices on Recommendation Systems, which breaks things down more along the lines of packages and implementations; probably what you want if you are an implementer not a researcher.

This area is moving fast, especially recently. I have the suspicion that the neural network methods are speciating at the moment.

Quentin Bacquet curtly summarises some methods and their performance on a problem, then introduces a VAE method via an implementation.

Ben Lorica in Questioning the Efficacy of Neural Recommendation Systems interviews Paolo Cremonesi and Maurizio Ferrari Dacrema, who are the authors of some recent review papers (Dacrema, Cremonesi, and Jannach 2019; Dacrema et al. 2021).

I am most familiar with the matrix factorization approaches (Koren, Bell, and Volinsky 2009; Yu et al. 2014, 2012; Lim and Teh 2007), especially NNMF) but there are many, e.g. variational autoencoder approaches are en vogue.

(Cheng et al. 2016) feels overcomplicated, but at least it is documented, and comes with an implementation: jrzaurin/pytorch-widedeep: A flexible package to combine tabular data with text and images using Wide and Deep models in Pytorch. Note that the author of that package advises using gradient boosting machines to get this job done.

Wu et al. (2021) taxonomises recommendation systems using Graph NNs, which is another natural idea. Their list goes a long way beyond Xiang Wang et al. (2019).

Tooling

Cremonesi and Dacrema maintain reproduction code for several popular algorithms which is probably pedagogically useful.

Microsoft Recommenders, already mentioned, includes basic wrappers for a lot of different tools, and also includes worked examples and deployment advice. Maybe start there.

torch has gone large here recently I think? At least if Introduction to TorchRec is anything to go by.

Vowpal wabbit

Vowpal wabbit implements some classic recommendation systems extremely efficiently, but the manual has historically been famously abstruse. Now it is merely gruff. There are, I think(?), two tutorials in the documentation:

There is a worked example, vowpal_wabbit_deep_dive.ipynb in Microsoft Recommenders.

For seduction and addiction of humans

Popular choice. extra material at clickbait bandits and recommender dynamics.

References

Abernethy, Jacob, Francis Bach, Theodoros Evgeniou, and Jean-Philippe Vert. 2009. β€œA New Approach to Collaborative Filtering: Operator Estimation with Spectral Regularization.” Journal of Machine Learning Research 10 (Mar): 803–26.
Argyriou, Andreas, Miguel GonzΓ‘lez-Fierro, and Le Zhang. 2020. β€œMicrosoft Recommenders: Best Practices for Production-Ready Recommendation Systems.” In Companion Proceedings of the Web Conference 2020, 50–51. New York, NY, USA: Association for Computing Machinery.
Brouwer, Thomas Alexander. 2017. β€œBayesian Matrix Factorisation: Inference, Priors, and Data Integration.”
Chen, Dawei, Lexing Xie, Aditya Krishna Menon, and Cheng Soon Ong. 2017. β€œStructured Recommendation.” arXiv:1706.09067 [Cs], June.
Cheng, Heng-Tze, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, et al. 2016. β€œWide & Deep Learning for Recommender Systems.” arXiv:1606.07792 [Cs, Stat], June.
Dacrema, Maurizio Ferrari, Simone Boglio, Paolo Cremonesi, and Dietmar Jannach. 2021. β€œA Troubling Analysis of Reproducibility and Progress in Recommender Systems Research.” ACM Transactions on Information Systems 39 (2): 1–49.
Dacrema, Maurizio Ferrari, Paolo Cremonesi, and Dietmar Jannach. 2019. β€œAre We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches.” In Proceedings of the 13th ACM Conference on Recommender Systems, 101–9. Copenhagen Denmark: ACM.
Davis III, Keith M., Michiel SpapΓ©, and Tuukka Ruotsalo. 2021. β€œCollaborative Filtering with Preferences Inferred from Brain Signals.” In Proceedings of the Web Conference 2021, 602–11. WWW ’21. New York, NY, USA: Association for Computing Machinery.
Dean, Sarah, and Jamie Morgenstern. 2022. β€œPreference Dynamics Under Personalized Recommendations.” arXiv.
Eilat, Itay, and Nir Rosenfeld. 2023. β€œPerformative Recommendation: Diversifying Content via Strategic Incentives.” arXiv.
FernΓ‘ndez, Miriam, Alejandro BellogΓ­n, and IvΓ‘n Cantador. 2021. β€œAnalysing the Effect of Recommendation Algorithms on the Amplification of Misinformation.” arXiv:2103.14748 [Cs], March.
Graham, Scott, Jun-Ki Min, and Tao Wu. 2019. β€œMicrosoft Recommenders: Tools to Accelerate Developing Recommender Systems.” In Proceedings of the 13th ACM Conference on Recommender Systems, 542–43. RecSys ’19. New York, NY, USA: Association for Computing Machinery.
He, Xiangnan, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. β€œNeural Collaborative Filtering.” In Proceedings of the 26th International Conference on World Wide Web, 173–82. WWW ’17. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee.
Heckerman, David, David Maxwell Chickering, Christopher Meek, Robert Rounthwaite, and Carl Kadie. 2000. β€œDependency Networks for Inference, Collaborative Filtering, and Data Visualization.” Journal of Machine Learning Research 1 (Oct): 49–75.
Hron, Jiri, Karl Krauth, Michael I. Jordan, Niki Kilbertus, and Sarah Dean. 2022. β€œModeling Content Creator Incentives on Algorithm-Curated Platforms.” arXiv.
Juan, Yuchin, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. β€œField-Aware Factorization Machines for CTR Prediction.” In Proceedings of the 10th ACM Conference on Recommender Systems, 43–50. RecSys ’16. New York, NY, USA: Association for Computing Machinery.
Koren, Yehuda, Robert Bell, and Chris Volinsky. 2009. β€œMatrix Factorization Techniques for Recommender Systems.” Computer 42 (8): 30–37.
Li, Lihong, Wei Chu, John Langford, and Xuanhui Wang. 2011. β€œUnbiased Offline Evaluation of Contextual-Bandit-Based News Article Recommendation Algorithms.” In Proceedings of the Fourth International Conference on Web Search and Web Data Mining (WSDM-11), 297–306.
Liang, Dawen, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. β€œVariational Autoencoders for Collaborative Filtering.” In Proceedings of the 2018 World Wide Web Conference, 689–98. WWW ’18. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee.
Lim, Yew Jin, and Yee Whye Teh. 2007. β€œVariational Bayesian Approach to Movie Rating Prediction.” In Proceedings of KDD Cup and Workshop, 7:15–21. Citeseer.
Liu, Yuli, Christian Walder, and Lexing Xie. 2022. β€œDeterminantal Point Process Likelihoods for Sequential Recommendation.” In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1653–63. SIGIR ’22. New York, NY, USA: Association for Computing Machinery.
Nicholson, Josh M., Milo Mordaunt, Patrice Lopez, Ashish Uppala, Domenic Rosati, Neves P. Rodrigues, Peter Grabitz, and Sean C. Rife. 2021. β€œScite: A Smart Citation Index That Displays the Context of Citations and Classifies Their Intent Using Deep Learning.” Quantitative Science Studies 2 (3): 882–98.
Raiko, Tapani, Alexander Ilin, and Juha Karhunen. 2007. β€œPrincipal Component Analysis for Large Scale Problems with Lots of Missing Values.” In Machine Learning: ECML 2007, edited by Joost N. Kok, Jacek Koronacki, Raomon Lopez de Mantaras, Stan Matwin, Dunja Mladenič, and Andrzej Skowron, 4701:691–98. Berlin, Heidelberg: Springer Berlin Heidelberg.
Rendle, Steffen. 2010. β€œFactorization Machines.” In 2010 IEEE International Conference on Data Mining, 995–1000.
Sachdeva, Noveen, Mehak Preet Dhaliwal, Carole-Jean Wu, and Julian McAuley. 2022. β€œInfinite Recommendation Networks: A Data-Centric Approach.” arXiv.
Sarwar, Badrul, George Karypis, Joseph Konstan, and John Riedl. 2002. β€œIncremental Singular Value Decomposition Algorithms for Highly Scalable Recommender Systems.”
Sedhain, Suvash, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015. β€œAutoRec: Autoencoders Meet Collaborative Filtering.” In Proceedings of the 24th International Conference on World Wide Web, 111–12. Florence Italy: ACM.
Sharma, Amit, Jake M. Hofman, and Duncan J. Watts. 2015. β€œEstimating the Causal Impact of Recommendation Systems from Observational Data.” Proceedings of the Sixteenth ACM Conference on Economics and Computation - EC ’15, 453–70.
Teeny, Jacob D., Joseph J. Siev, Pablo BriΓ±ol, and Richard E. Petty. 2021. β€œA Review and Conceptual Framework for Understanding Personalized Matching Effects in Persuasion.” Journal of Consumer Psychology 31 (2): 382–414.
Wang, Xiang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. β€œNeural Graph Collaborative Filtering.” In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 165–74. SIGIR’19. New York, NY, USA: Association for Computing Machinery.
Wang, Xinxi, and Ye Wang. 2014. β€œImproving Content-Based and Hybrid Music Recommendation Using Deep Learning.” In Proceedings of the 22Nd ACM International Conference on Multimedia, 627–36. MM ’14. New York, NY, USA: ACM.
Whittaker, Joe, SeΓ‘n Looney, Alastair Reed, and Fabio Votta. 2021. β€œRecommender Systems and the Amplification of Extremist Content.” Internet Policy Review 10 (2).
Wu, Shiwen, Fei Sun, Wentao Zhang, and Bin Cui. 2021. β€œGraph Neural Networks in Recommender Systems: A Survey.” arXiv:2011.02260 [Cs], April.
Xia, Min, Xianwen Yu, Xiaoning Zhang, and Yang Cao. 2019. β€œVAEGAN: A Collaborative Filtering Framework Based on Adversarial Variational Autoencoders,” 4206–12.
Yu, Hsiang-Fu, Cho-Jui Hsieh, Si Si, and Inderjit S. Dhillon. 2012. β€œScalable Coordinate Descent Approaches to Parallel Matrix Factorization for Recommender Systems.” In IEEE International Conference of Data Mining, 765–74.
β€”β€”β€”. 2014. β€œParallel Matrix Factorization for Recommender Systems.” Knowledge and Information Systems 41 (3): 793–819.
Zhao, Zhe, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. β€œRecommending What Video to Watch Next: A Multitask Ranking System.” In Proceedings of the 13th ACM Conference on Recommender Systems, 43–51. RecSys ’19. New York, NY, USA: Association for Computing Machinery.

Download bibliography

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.