External validity

TBD.

One could read Sebastian Ruder’s NN-style introduction to “transfer learning”. NN people like to think about this in particular way which I like because of the diversity of out-of-the-box ideas it invites and which I dislike because it is sloppy. The central idea here is learning well-factored causal graphical models and everything else is just an approximation to that. The reason this is hot topic in neural nets, I suspect, is that it is convenient for massive, low-human-effort neural networks to ignore graphical structure to get good results from regressions in observational data. To recovery the fancy performance in a black-box model is even more tedious than a classical one. Also it fits the social conventions of neural network research to reinvent methods to fix such problems without reference to previous conventions, for better and worse.

One thing that the machine learning set up gives us which is an additional emphasis: external validity, the traditional framing, would ask you whether the model you have learnt is still useful on new data. The transfer learning set up invites use to consider if we can transfer some of the computational effort from learning on one data set to learning on new dataset, and if so, how much.

This connects also to semi-supervised learning and fairness, argues (Bernhard Schölkopf Bernhard et al. 2012; Bernhard Schölkopf 2019).