# optimization

Stein’s method
2021-03-12
– 2021-05-12
Probability divergences
Metrics, contrasts and divergences and other ways of quantifying how similar are two randomnesses
2014-11-25
– 2021-05-12
Differentiable model selection
Differentiable hyperparameter search, and architecture search, and optimisation optimisation by optimisation and so on
2020-09-25
– 2021-04-13
Generically approximating probability distributions
2021-03-12
– 2021-03-22
Optimal transport inference
I feel the earth mover under my feet, I feel the ψ tumbling down, I feel my heart start to trembling, Whenever you're around my empirical density in minimal transport cost
2021-03-16
Optimal transport metrics
Wasserstein distances, Monge-Kantorovich metrics, Earthmover distances
2019-05-30
– 2021-03-16
ML scaling laws in the massive model limit
2021-01-14
– 2021-03-15
Neural nets with implicit layers
Also, declarative networks
2020-12-08
– 2021-03-15
Neural nets with basis decomposition layers
2021-03-09
Reparameterization tricks in inference
Normalizing flows, invertible density models, inference by measure transport, low-dimensional coupling…
2018-04-04
– 2021-03-08
Automatic differentiation
2016-07-27
– 2021-03-08
Hyperparameter optimization in ML
Replacing a hyperparameter problem with a hyperhyperparameter problem which feels like progress I guess
2020-09-25
– 2021-03-01
Jax
2020-09-15
– 2021-02-18
Random-forest-like methods
Boosting, bagging and other weak-learner ensemble methods
2015-09-23
– 2021-02-11
Wiener-Khintchine representation
Now with bonus Bochner!
2019-05-08
– 2021-01-03
Distribution regression
2020-12-01
Recommender systems
2020-11-30
Variational inference by message-passing in graphical models
2014-11-25
– 2020-11-25
Tensorflow
The framework to use for deep learning if you groupthink like Google
2016-07-11
– 2020-11-10
ELBO
Evidence lower bound, variational free energy etc
2020-10-02
– 2020-10-28
Gradient descent
First order of business
2014-10-04
– 2020-10-27
Efficient factoring of GP likelihoods
2020-10-16
– 2020-10-26
Differentiating through the Gamma
2020-06-12
– 2020-10-15
Automatic design of experiments
Minesweeper++
2017-04-11
– 2020-10-13
AutoML
2017-07-17
– 2020-10-02
Data summarization
On maps drawn at smaller than 1:1 scale
2019-01-14
– 2020-09-18
Variational autoencoders
2019-11-04
– 2020-09-10
Online learning
2018-09-30
– 2020-08-26
(Outlier) robust statistics
2014-11-25
– 2020-07-14
Automatic differentiation in Julia
2016-07-27
– 2020-06-05
Voice fakes
2018-09-06
– 2020-05-27
Variational inference
On fitting the best model one can be bothered to
2016-03-22
– 2020-05-24
Matrix calculus
2018-07-09
– 2020-05-19
Evolution
2014-11-04
– 2020-04-27
Gradient descent, first-order, stochastic
a.k.a. SGD, as seen in deep learning
2020-01-30
– 2020-02-07
Gradient descent, Newton-like, stochastic
2020-01-23
Phase retrieval
I’ve got the power. / Like the crack of the whip/ I snap attack/ Front to back
2017-01-16
– 2019-11-07
Sparse coding
How to make big things out of lists of small things.
2014-11-17
– 2019-11-05
Optimal control
2015-06-22
– 2019-11-01
Gradient descent, Higher order
2019-10-26
Statistical learning theory for time series
2016-11-03
– 2019-10-01
Wirtinger calculus
It’s not complicated / It’s complex
2019-05-08
– 2019-09-10
Large sample theory
2015-02-15
– 2019-09-09
Gradient descent, Newton-like
2019-02-05
– 2019-09-03
Semidefinite proramming
2019-06-29
Optimisation
2014-10-04
– 2019-06-27
(Weighted) least squares fits
2016-09-22
– 2019-05-22
Wiener theorem
Now with bonus Bochner!
2019-05-08
Wacky regression
2015-09-23
– 2019-05-02
Nearly sufficient statistics
How about “Sufficient sufficiency”? — is that taken?
2018-03-13
– 2019-01-14
Variational state filtering
2018-03-19
– 2018-12-07
Optimisation, combinatorial
2018-08-11
Submodular functions, maximizing
2018-07-09
Gradient descent, continuous, primal/dual formulations.
2017-08-07
Lagrangian mechanics
2015-02-11
– 2017-06-18
Maximum likelihood inference
2015-02-15
– 2016-10-13
Distributed optimization for regression
2016-10-11
M-estimation
2016-07-11
– 2016-10-10
Generalised linear models
2016-03-24
– 2016-08-31
Statistical learning theory
Eventually including structural risk minimisation, risk bounds, hopefully-uniform convergence rates, VC-dimension, generalisation-and-stability framings etc
2016-07-06
– 2016-08-16
Probably Approximately Correct
2014-11-24
– 2016-05-29
Expectation maximisation
2014-08-17
– 2016-04-17
Biomimetic algorithms
2015-12-22
Geometry of fitness landscapes
2011-07-27
– 2015-06-17