Model selection

Hyperparameter optimization Replacing a hyperparameter problem with a hyperhyperparameter problem which feels like progress I guess 2020-09-25 – 2021-10-20
Meta learning Few-shot learning, learning fast weights, learning to learn 2021-09-16
Compressing neural nets pruning, compacting and otherwise fitting a good estimate into fewer parameters 2016-10-14 – 2021-05-07
Differentiable model selection Differentiable hyperparameter search, and architecture search, and optimisation optimisation by optimisation and so on 2020-09-25 – 2021-04-13
AutoML 2017-07-17 – 2020-10-02
Model averaging On keeping many incorrect hypotheses and using them all as one goodish one 2017-06-20 – 2020-03-22
Statistical learning theory Eventually including structural risk minimisation, risk bounds, hopefully-uniform convergence rates, VC-dimension, generalisation-and-stability framings etc 2016-07-06 – 2016-08-16