Recommender systems

November 30, 2020 — August 15, 2022

high d
linear algebra
probabilistic algorithms
sparser than thou

Assumed audience:

People who have not heard about recommender systems yet

Not my core area, but I need a landing page to refer to because this is coming up in conversations a lot at the moment.

Figure 1: Users who liked apostasy also enjoyed…

Recommender systems are a weird area. There have been some high profile examples of recommender models going back a long time (e.g. the Netflix Prize) and they are clearly commercially important. At time of writing, though, there is relatively little material introducing these systems to wide audiences, compared to say linear regressions and image classifiers and so on. Possibly this is something about who owns the data to make these tools go, which is to say, there are relatively few open data sets for recommender systems?

I would like to refer to a simple and good introduction to the area, but I do not know one, so what follows a scruffy word salad with some helpful links.

YouTube’s defence of their recommender system introduces in vague outline a lot of the features that make it go on the way to defending them from allegations of feeding extremism [citation needed].

1 Zoo

Starting from an overview by Javier we can find many approaches to more abstract recommender systems.

  1. Most Popular recommendations (the baseline)
  2. Item-User similarity based recommendations
  3. kNN Collaborative Filtering recommendations
  4. Gradient Boosting Machine based recommendations
  5. Non-Negative Matrix Factorization recommendations
  6. Factorization Machines (Rendle 2010)
  7. Field Aware Factorization Machines (Juan et al. 2016)
  8. Deep Learning based recommendations (Wide and Deep, Cheng et al. (2016))
  9. Neural Collaborative Filtering (He et al. 2017)
  10. Neural Graph Collaborative Filtering (Xiang Wang et al. 2019)
  11. Variational Autoencoders for Collaborative Filtering (Liang et al. 2018)

Not mentioned in that list filtering based on monitoring the brains of subjects (Davis III, Spapé, and Ruotsalo 2021).

A different take is in Microsoft’s Best Practices on Recommendation Systems, which breaks things down more along the lines of packages and implementations; probably what you want if you are an implementer not a researcher.

This area is moving fast, especially recently. I have the suspicion that the neural network methods are speciating at the moment.

Quentin Bacquet curtly summarises some methods and their performance on a problem, then introduces a VAE method via an implementation.

Ben Lorica in Questioning the Efficacy of Neural Recommendation Systems interviews Paolo Cremonesi and Maurizio Ferrari Dacrema, who are the authors of some recent review papers (Dacrema, Cremonesi, and Jannach 2019; Dacrema et al. 2021).

I am most familiar with the matrix factorization approaches (Koren, Bell, and Volinsky 2009; Yu et al. 2014, 2012; Lim and Teh 2007), especially matrix_factorisation_nonneg) but there are many, e.g. variational autoencoder approaches are en vogue.

(Cheng et al. 2016) feels overcomplicated, but at least it is documented, and comes with an implementation: jrzaurin/pytorch-widedeep: A flexible package to combine tabular data with text and images using Wide and Deep models in Pytorch. Note that the author of that package advises using gradient boosting machines to get this job done.

Wu et al. (2021) taxonomises recommendation systems using Graph NNs, which is another natural idea. Their list goes a long way beyond Xiang Wang et al. (2019).

2 Tooling

Cremonesi and Dacrema maintain reproduction code for several popular algorithms which is probably pedagogically useful.

Microsoft Recommenders, already mentioned, includes basic wrappers for a lot of different tools, and also includes worked examples and deployment advice. Maybe start there.

torch has gone large here recently I think? At least if Introduction to TorchRec is anything to go by.

2.1 Vowpal wabbit

Vowpal wabbit implements some classic recommendation systems extremely efficiently, but the manual has historically been famously abstruse. Now it is merely gruff. There are, I think(?), two tutorials in the documentation:

There is a worked example, vowpal_wabbit_deep_dive.ipynb in Microsoft Recommenders.

3 For seduction and addiction of humans

Popular choice. extra material at clickbait bandits and recommender dynamics.

4 References

Abernethy, Bach, Evgeniou, et al. 2009. A New Approach to Collaborative Filtering: Operator Estimation with Spectral Regularization.” Journal of Machine Learning Research.
Argyriou, González-Fierro, and Zhang. 2020. Microsoft Recommenders: Best Practices for Production-Ready Recommendation Systems.” In Companion Proceedings of the Web Conference 2020.
Brouwer. 2017. Bayesian Matrix Factorisation: Inference, Priors, and Data Integration.”
Cheng, Koc, Harmsen, et al. 2016. Wide & Deep Learning for Recommender Systems.” arXiv:1606.07792 [Cs, Stat].
Chen, Xie, Menon, et al. 2017. Structured Recommendation.” arXiv:1706.09067 [Cs].
Dacrema, Boglio, Cremonesi, et al. 2021. A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research.” ACM Transactions on Information Systems.
Dacrema, Cremonesi, and Jannach. 2019. Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches.” In Proceedings of the 13th ACM Conference on Recommender Systems.
Davis III, Spapé, and Ruotsalo. 2021. Collaborative Filtering with Preferences Inferred from Brain Signals.” In Proceedings of the Web Conference 2021. WWW ’21.
Dean, and Morgenstern. 2022. Preference Dynamics Under Personalized Recommendations.”
Eilat, and Rosenfeld. 2023. Performative Recommendation: Diversifying Content via Strategic Incentives.”
Fernández, Bellogín, and Cantador. 2021. Analysing the Effect of Recommendation Algorithms on the Amplification of Misinformation.” arXiv:2103.14748 [Cs].
Graham, Min, and Wu. 2019. Microsoft Recommenders: Tools to Accelerate Developing Recommender Systems.” In Proceedings of the 13th ACM Conference on Recommender Systems. RecSys ’19.
Heckerman, Chickering, Meek, et al. 2000. Dependency Networks for Inference, Collaborative Filtering, and Data Visualization.” Journal of Machine Learning Research.
He, Liao, Zhang, et al. 2017. Neural Collaborative Filtering.” In Proceedings of the 26th International Conference on World Wide Web. WWW ’17.
Hron, Krauth, Jordan, et al. 2022. Modeling Content Creator Incentives on Algorithm-Curated Platforms.”
Juan, Zhuang, Chin, et al. 2016. Field-Aware Factorization Machines for CTR Prediction.” In Proceedings of the 10th ACM Conference on Recommender Systems. RecSys ’16.
Koren, Bell, and Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems.” Computer.
Liang, Krishnan, Hoffman, et al. 2018. Variational Autoencoders for Collaborative Filtering.” In Proceedings of the 2018 World Wide Web Conference. WWW ’18.
Li, Chu, Langford, et al. 2011. Unbiased Offline Evaluation of Contextual-Bandit-Based News Article Recommendation Algorithms.” In Proceedings of the Fourth International Conference on Web Search and Web Data Mining (WSDM-11).
Lim, and Teh. 2007. “Variational Bayesian Approach to Movie Rating Prediction.” In Proceedings of KDD Cup and Workshop.
Liu, Walder, and Xie. 2022. Determinantal Point Process Likelihoods for Sequential Recommendation.” In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR ’22.
Nicholson, Mordaunt, Lopez, et al. 2021. Scite: A Smart Citation Index That Displays the Context of Citations and Classifies Their Intent Using Deep Learning.” Quantitative Science Studies.
Raiko, Ilin, and Karhunen. 2007. Principal Component Analysis for Large Scale Problems with Lots of Missing Values.” In Machine Learning: ECML 2007.
Rendle. 2010. Factorization Machines.” In 2010 IEEE International Conference on Data Mining.
Sachdeva, Dhaliwal, Wu, et al. 2022. Infinite Recommendation Networks: A Data-Centric Approach.”
Sarwar, Karypis, Konstan, et al. 2002. “Incremental Singular Value Decomposition Algorithms for Highly Scalable Recommender Systems.”
Sedhain, Menon, Sanner, et al. 2015. AutoRec: Autoencoders Meet Collaborative Filtering.” In Proceedings of the 24th International Conference on World Wide Web.
Sharma, Hofman, and Watts. 2015. Estimating the Causal Impact of Recommendation Systems from Observational Data.” Proceedings of the Sixteenth ACM Conference on Economics and Computation - EC ’15.
Teeny, Siev, Briñol, et al. 2021. A Review and Conceptual Framework for Understanding Personalized Matching Effects in Persuasion.” Journal of Consumer Psychology.
Wang, Xiang, He, Wang, et al. 2019. Neural Graph Collaborative Filtering.” In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR’19.
Wang, Xinxi, and Wang. 2014. Improving Content-Based and Hybrid Music Recommendation Using Deep Learning.” In Proceedings of the 22Nd ACM International Conference on Multimedia. MM ’14.
Whittaker, Looney, Reed, et al. 2021. Recommender Systems and the Amplification of Extremist Content.” Internet Policy Review.
Wu, Sun, Zhang, et al. 2021. Graph Neural Networks in Recommender Systems: A Survey.” arXiv:2011.02260 [Cs].
Xia, Yu, Zhang, et al. 2019. VAEGAN: A Collaborative Filtering Framework Based on Adversarial Variational Autoencoders.”
Xu, Tan, Fu, et al. 2022. Dynamic Causal Collaborative Filtering.” In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. CIKM ’22.
Yu, Hsieh, Si, et al. 2012. Scalable Coordinate Descent Approaches to Parallel Matrix Factorization for Recommender Systems.” In IEEE International Conference of Data Mining.
———, et al. 2014. Parallel Matrix Factorization for Recommender Systems.” Knowledge and Information Systems.
Zhao, Hong, Wei, et al. 2019. Recommending What Video to Watch Next: A Multitask Ranking System.” In Proceedings of the 13th ACM Conference on Recommender Systems. RecSys ’19.