optimization

Meta learning Few-shot learning, learning fast weights, learning to learn 2021-09-16

pytorch #torched 2018-05-04 – 2021-09-14

Wirtinger calculus It’s not complicated / It’s complex 2019-05-08 – 2021-09-07

Neural nets with implicit layers Also, declarative networks 2020-12-08 – 2021-09-07

Automatic differentiation in Julia 2016-07-27 – 2021-08-27

Wiener-Khintchine representations Spectral representations of stochastic processes 2019-05-08 – 2021-08-23

Automatic differentiation 2016-07-27 – 2021-08-05

Generalised linear models 2016-03-24 – 2021-08-05

Sequential experiments Especially multiple sequential experiments 2021-08-04

Jax 2020-09-15 – 2021-07-30

Recommender systems 2020-11-30 – 2021-07-19

Gradient descent at scale 2021-07-14

Tensorflow The framework to use for deep learning if you groupthink like Google 2016-07-11 – 2021-07-07

Learning on tabular data 2020-11-30 – 2021-06-21

Random-forest-like methods A selection of randomly stopped clocks is never far from wrong. 2015-09-23 – 2021-06-17

Voice fakes 2018-09-06 – 2021-06-17

Optimal transport metrics Wasserstein distances, Monge-Kantorovich metrics, Earthmover distances 2019-05-30 – 2021-06-08

Deep generative models 2020-12-10 – 2021-06-08

Energy based models Inference with kinda-tractable un-normalized densities 2021-06-07

Stein’s method 2021-03-12 – 2021-06-01

Probability divergences Metrics, contrasts and divergences and other ways of quantifying how similar are two randomnesses 2014-11-25 – 2021-05-12

Differentiable model selection Differentiable hyperparameter search, and architecture search, and optimisation optimisation by optimisation and so on 2020-09-25 – 2021-04-13

Generically approximating probability distributions 2021-03-12 – 2021-03-22

Optimal transport inference I feel the earth mover under my feet, I feel the ψ tumbling down, I feel my heart start to trembling, Whenever you’re around my empirical density in minimal transport cost 2021-03-16

Scaling laws for very large neural nets 2021-01-14 – 2021-03-15

Neural nets with basis decomposition layers 2021-03-09

Reparameterization tricks in inference Normalizing flows, invertible density models, inference by measure transport, low-dimensional coupling… 2018-04-04 – 2021-03-08

Hyperparameter optimization in ML Replacing a hyperparameter problem with a hyperhyperparameter problem which feels like progress I guess 2020-09-25 – 2021-03-01

Generative adversarial networks 2016-10-07 – 2020-12-14

Distribution regression 2020-12-01

Variational inference by message-passing in graphical models 2014-11-25 – 2020-11-25

ELBO Evidence lower bound, variational free energy etc 2020-10-02 – 2020-10-28

Gradient descent First order of business 2014-10-04 – 2020-10-27

Efficient factoring of GP likelihoods 2020-10-16 – 2020-10-26

Differentiating through the Gamma 2020-06-12 – 2020-10-15

Automatic design of experiments Minesweeper++ 2017-04-11 – 2020-10-13

AutoML 2017-07-17 – 2020-10-02

Data summarization On maps drawn at smaller than 1:1 scale 2019-01-14 – 2020-09-18

Variational autoencoders 2019-11-04 – 2020-09-10

Online learning 2018-09-30 – 2020-08-26

(Outlier) robust statistics 2014-11-25 – 2020-07-14

Variational inference On fitting the best model one can be bothered to 2016-03-22 – 2020-05-24

Matrix calculus 2018-07-09 – 2020-05-19

Evolution 2014-11-04 – 2020-04-27

Gradient descent, first-order, stochastic a.k.a. SGD, as seen in deep learning 2020-01-30 – 2020-02-07

Gradient descent, Newton-like, stochastic 2020-01-23

Phase retrieval I’ve got the power. / Like the crack of the whip/ I snap attack/ Front to back 2017-01-16 – 2019-11-07

Sparse coding How to make big things out of lists of small things. 2014-11-17 – 2019-11-05

Optimal control 2015-06-22 – 2019-11-01

Gradient descent, Higher order 2019-10-26

Statistical learning theory for time series 2016-11-03 – 2019-10-01

Large sample theory 2015-02-15 – 2019-09-09

Gradient descent, Newton-like 2019-02-05 – 2019-09-03

Semidefinite proramming 2019-06-29

Optimisation 2014-10-04 – 2019-06-27

(Weighted) least squares fits 2016-09-22 – 2019-05-22

Wiener theorem Now with bonus Bochner! 2019-05-08

Wacky regression 2015-09-23 – 2019-05-02

Nearly sufficient statistics How about “Sufficient sufficiency”? — is that taken? 2018-03-13 – 2019-01-14

Variational state filtering 2018-03-19 – 2018-12-07

Optimisation, combinatorial 2018-08-11

Submodular functions, maximizing 2018-07-09

Gradient descent, continuous, primal/dual formulations. 2017-08-07

Lagrangian mechanics 2015-02-11 – 2017-06-18

Maximum likelihood inference 2015-02-15 – 2016-10-13

Distributed optimization for regression 2016-10-11

M-estimation 2016-07-11 – 2016-10-10

Statistical learning theory Eventually including structural risk minimisation, risk bounds, hopefully-uniform convergence rates, VC-dimension, generalisation-and-stability framings etc 2016-07-06 – 2016-08-16

Probably Approximately Correct 2014-11-24 – 2016-05-29

Expectation maximisation 2014-08-17 – 2016-04-17

Biomimetic algorithms 2015-12-22

Geometry of fitness landscapes 2011-07-27 – 2015-06-17