optimization
Pytorch
#torched
2018-05-04
– 2022-05-18Scaling laws for very large neural nets
2021-01-14
– 2022-05-12Smooth transforms of Gaussian noise
Delta method, error propagation, unscented transform, Taylor expansion, in finite and inifinite dimensional spaces
2014-11-25
– 2022-05-12SLAM
Simultaneous Location and Mapping
2014-11-25
– 2022-04-28Gradient descent
First order of business
2014-10-04
– 2022-04-28Vecchia factoring of GP likelihoods
Ignore some conditioning in the dependencies and attain a sparse cholesky factor for the precision matrix
2022-04-27Jax
Julia for python
2020-09-15
– 2022-04-23Basis-functions in Gaussian process regression
a.k.a Fixed Rank Kriging, basis function regression, weight space, spatial random effects
2022-02-22
– 2022-04-21Particle belief propagation
Graphical inference using empirical distribution estimates
2014-07-25
– 2022-04-08Particle Markov Chain Monte Carlo
Particle systems as MCMC proposals
2014-07-25
– 2022-04-08Matrix calculus
2018-07-09
– 2022-04-04Belief propagation
2014-11-25
– 2022-03-31Gaussian belief propagation
Least squares at maximal elaboration
2014-11-25
– 2022-03-28Wiener-Khintchine representations
Spectral representations of stochastic processes
2019-05-08
– 2022-03-11Neural nets with implicit layers
Also, declarative networks
2020-12-08
– 2022-03-09Automatic differentiation in Julia
2016-07-27
– 2022-03-09Genetic programming
2015-12-22
– 2022-03-09Sparse coding
Wavelets, matching pursuit, overcomplete dictionaries…
2014-11-17
– 2022-03-07Generative flow
2022-03-07Optimal transport inference
I feel the earth mover under my feet, I feel the ψ tumbling down, I feel my heart start to trembling, Whenever you’re around my empirical density in minimal transport cost
2021-03-16
– 2022-02-22M-estimation
2016-07-11
– 2022-02-17Message-passing algorithms in graphical models
Cleaving reality at the joint, then summing it at the marginal
2014-11-25
– 2022-02-17Variational inference
On fitting something not too far from a pretty good model that is not too hard
2016-03-22
– 2022-02-10Evolution
2014-11-04
– 2022-02-07Recommender systems
2020-11-30
– 2022-02-04Neural nets with basis decomposition layers
2021-03-09
– 2022-02-01Running neural nets backwards
2022-01-29Machine learning and statistics in Julia
2019-11-27
– 2022-01-27(Outlier) robust statistics
2014-11-25
– 2022-01-21Reparameterization tricks in inference
Pathwise gradient estimation, nNormalizing flows, invertible density models, inference by measure transport, low-dimensional coupling…
2018-04-04
– 2021-12-21Gradient descent, Newton-like, stochastic
2020-01-23
– 2021-12-09Variational state filtering
2018-03-19
– 2021-12-08Gaussian Processes as stochastic differential equations
Imposing time on things
2019-09-18
– 2021-11-25Optimisation
2014-10-04
– 2021-11-11Neural diffusion models
2021-11-11
– 2021-11-11Deep generative models
2020-12-10
– 2021-11-11Probably Approximately Correct
2014-11-24
– 2021-10-27Hyperparameter optimization
Replacing a hyperparameter problem with a hyperhyperparameter problem which feels like progress I guess
2020-09-25
– 2021-10-20Gradient descent at scale
Practical implementation of large optimisations
2021-07-14
– 2021-09-28Meta learning
Few-shot learning, learning fast weights, learning to learn
2021-09-16Wirtinger calculus
It’s not complicated / It’s complex
2019-05-08
– 2021-09-07Automatic differentiation
2016-07-27
– 2021-08-05Generalised linear models
2016-03-24
– 2021-08-05Sequential experiments
Especially multiple sequential experiments
2021-08-04Tensorflow
The framework to use for deep learning if you groupthink like Google
2016-07-11
– 2021-07-07Learning on tabular data
2020-11-30
– 2021-06-21Random-forest-like methods
A selection of randomly stopped clocks is never far from wrong.
2015-09-23
– 2021-06-17Voice fakes
2018-09-06
– 2021-06-17Optimal transport metrics
Wasserstein distances, Monge-Kantorovich metrics, Earthmover distances
2019-05-30
– 2021-06-08Energy based models
Inference with kinda-tractable un-normalized densities
2021-06-07Stein’s method
2021-03-12
– 2021-06-01Probability divergences
Metrics, contrasts and divergences and other ways of quantifying how similar are two randomnesses
2014-11-25
– 2021-05-12Differentiable model selection
Differentiable hyperparameter search, and architecture search, and optimisation optimisation by optimisation and so on
2020-09-25
– 2021-04-13Generically approximating probability distributions
2021-03-12
– 2021-03-22Generative adversarial networks
2016-10-07
– 2020-12-14Distribution regression
2020-12-01ELBO
Evidence lower bound, variational free energy etc
2020-10-02
– 2020-10-28Efficient factoring of GP likelihoods
2020-10-16
– 2020-10-26Automatic design of experiments
Minesweeper++
2017-04-11
– 2020-10-13AutoML
2017-07-17
– 2020-10-02Data summarization
On maps drawn at smaller than 1:1 scale
2019-01-14
– 2020-09-18Variational autoencoders
2019-11-04
– 2020-09-10Online learning
2018-09-30
– 2020-08-26Variational inference
On fitting the best model one can be bothered to
2016-03-22
– 2020-05-24Gradient descent, first-order, stochastic
a.k.a. SGD, as seen in deep learning
2020-01-30
– 2020-02-07Phase retrieval
I’ve got the power. / Like the crack of the whip/ I snap attack/ Front to back
2017-01-16
– 2019-11-07Optimal control
2015-06-22
– 2019-11-01Gradient descent, Higher order
2019-10-26Statistical learning theory for time series
2016-11-03
– 2019-10-01Large sample theory
2015-02-15
– 2019-09-09Gradient descent, Newton-like
2019-02-05
– 2019-09-03Semidefinite proramming
2019-06-29(Weighted) least squares fits
2016-09-22
– 2019-05-22Wiener theorem
Now with bonus Bochner!
2019-05-08Wacky regression
2015-09-23
– 2019-05-02Nearly sufficient statistics
How about “Sufficient sufficiency”? — is that taken?
2018-03-13
– 2019-01-14Optimisation, combinatorial
2018-08-11Submodular functions, maximizing
2018-07-09Gradient descent, continuous, primal/dual formulations.
2017-08-07Lagrangian mechanics
2015-02-11
– 2017-06-18Maximum likelihood inference
2015-02-15
– 2016-10-13Distributed statistica inference
2016-10-11Statistical learning theory
Eventually including structural risk minimisation, risk bounds, hopefully-uniform convergence rates, VC-dimension, generalisation-and-stability framings etc
2016-07-06
– 2016-08-16Expectation maximisation
2014-08-17
– 2016-04-17Biomimetic algorithms
2015-12-22Geometry of fitness landscapes
2011-07-27
– 2015-06-17