optimization

Hyperparameter optimization in ML Replacing a hyperparameter problem with a hyperhyperparameter problem which feels like progress I guess 2020-09-25 – 2021-03-01

Jax 2020-09-15 – 2021-02-18

Random-forest-like methods Boosting, bagging and other weak-learner ensemble methods 2015-09-23 – 2021-02-11

ML scaling in the massive parameter limit 2021-01-14

Wiener-Khintchine representation Now with bonus Bochner! 2019-05-08 – 2021-01-03

Reparameterization tricks in inference Normalizing flows, invertible density models, inference by measure transport, low-dimensional coupling… 2018-04-04 – 2020-12-18

Automatic differentiation 2016-07-27 – 2020-12-11

Distribution regression 2020-12-01

Recommender systems 2020-11-30

Variational inference by message-passing in graphical models 2014-11-25 – 2020-11-25

Tensorflow The framework to use for deep learning if you groupthink like Google 2016-07-11 – 2020-11-10

ELBO Evidence lower bound, variational free energy etc 2020-10-02 – 2020-10-28

Gradient descent First order of business 2014-10-04 – 2020-10-27

Efficient factoring of GP likelihoods 2020-10-16 – 2020-10-26

Differentiating through the Gamma 2020-06-12 – 2020-10-15

Automatic design of experiments Minesweeper++ 2017-04-11 – 2020-10-13

AutoML 2017-07-17 – 2020-10-02

Data summarization On maps drawn at smaller than 1:1 scale 2019-01-14 – 2020-09-18

Variational autoencoders 2019-11-04 – 2020-09-10

Online learning 2018-09-30 – 2020-08-26

(Outlier) robust statistics 2014-11-25 – 2020-07-14

Automatic differentiation in Julia 2016-07-27 – 2020-06-05

Voice fakes 2018-09-06 – 2020-05-27

Variational inference On fitting the best model one can be bothered to 2016-03-22 – 2020-05-24

Matrix calculus 2018-07-09 – 2020-05-19

Evolution 2014-11-04 – 2020-04-27

Optimal transport metrics Wasserstein distances, Monge-Kantorovich metrics, Earthmover distances 2019-05-30 – 2020-03-06

Gradient descent, first-order, stochastic a.k.a. SGD, as seen in deep learning 2020-01-30 – 2020-02-07

Gradient descent, Newton-like, stochastic 2020-01-23

Phase retrieval I’ve got the power. / Like the crack of the whip/ I snap attack/ Front to back 2017-01-16 – 2019-11-07

Sparse coding How to make big things out of lists of small things. 2014-11-17 – 2019-11-05

Optimal control 2015-06-22 – 2019-11-01

Gradient descent, Higher order 2019-10-26

Statistical learning theory for time series 2016-11-03 – 2019-10-01

Wirtinger calculus It’s not complicated / It’s complex 2019-05-08 – 2019-09-10

Large sample theory 2015-02-15 – 2019-09-09

Gradient descent, Newton-like 2019-02-05 – 2019-09-03

Semidefinite proramming 2019-06-29

Optimisation 2014-10-04 – 2019-06-27

(Weighted) least squares fits 2016-09-22 – 2019-05-22

Wiener theorem Now with bonus Bochner! 2019-05-08

Wacky regression 2015-09-23 – 2019-05-02

Nearly sufficient statistics How about “Sufficient sufficiency”? — is that taken? 2018-03-13 – 2019-01-14

Variational state filtering 2018-03-19 – 2018-12-07

Optimisation, combinatorial 2018-08-11

Submodular functions, maximizing 2018-07-09

Gradient descent, continuous, primal/dual formulations. 2017-08-07

Lagrangian mechanics 2015-02-11 – 2017-06-18

Maximum likelihood inference 2015-02-15 – 2016-10-13

Distributed optimization for regression 2016-10-11

M-estimation 2016-07-11 – 2016-10-10

Generalised linear models 2016-03-24 – 2016-08-31

Statistical learning theory Eventually including structural risk minimisation, risk bounds, hopefully-uniform convergence rates, VC-dimension, generalisation-and-stability framings etc 2016-07-06 – 2016-08-16

Probably Approximately Correct 2014-11-24 – 2016-05-29

Expectation maximisation 2014-08-17 – 2016-04-17

Biomimetic algorithms 2015-12-22

Geometry of fitness landscapes 2011-07-27 – 2015-06-17