Probably actually reading

gd

• The zen of gradient descent
• Optimization I | Simons Institute for the Theory of Computing
• Computer Scientists Discover Limits of Major Research Algorithm | Quanta Magazine

Message passing

• From automatic differentiation to message passing

Music

Nestup / cutelabnyc/nested-tuplets: Fancy javascript for manipulating nested tuplets.

Foundations

• Identity Crisis
• Invited Talk - Michael Bronstein · SlidesLive
• Product Placement
• (Not So) Free Samples | Michael Betancourt on Patreon
• Sampling Case Study Live Stream Wed 4/14 at 2 PM EDT | Michael Betancourt on Patreon
• Updated Geometric Optimization Paper | Michael Betancourt on Patreon
• We Built Sparse City | Michael Betancourt on Patreon
• Modeling Sparsity Case Study Live Stream Tue 6/8 at 2 PM EDT | Michael Betancourt on Patreon

GP research

• Bayesian inference with INLA
• R-INLA Project
• Linear Models from a Gaussian Process Point of View with Stheno and JAX · Invenia Blog
• Regression-based covariance functions for nonstationary spatial modeling
• kalman-jax/sde_gp.py at master · AaltoML/kalman-jax
• Probability Theory (For Scientists and Engineers)
• Scaling multi-output Gaussian process models with exact inference · Invenia Blog
• wesselb/stheno: Gaussian process modelling in Python

D R MacIver

I have been enjoying some of the explanatory style of David R MacIver recently.

• Overthinking Everything
• DRMacIver’s Notebook: The trauma model of talent blocks
• Jiminy Cricket Must Die | David R. MacIver
• Gendering | David R. MacIver
• Best of drmaciver.com | David R. MacIver
• DRMacIver’s Notebook: Emotional reactions as legacy code
• DRMacIver’s Notebook: Your emotions are valid but probably wrong

Misc

• How to write a great research paper - Microsoft Research
• AaltoML/kalman-jax: Approximate inference for Markov Gaussian processes using iterated Kalman smoothing, in JAX
• Cheng Soon Ong, Marc Peter Deisenroth | There and Back Again: A Tale of Slopes and Expectations · SlidesLive
• David Duvenaud, J. Zico Kolter, Matt Johnson | Deep Implicit Layers: Neural ODEs, Equilibrium Models and Beyond · SlidesLive
• Overview · ADCME
• Encoder Autonomy | Machine Thoughts
• The Notion of “Double Descent” | Mad (Data) Scientist
• pyro SVI
• Jaan on translating between variational terminology in physics and ML
• Jaan on VAE
• pamamakouros on normalizing flows
• Eric Jang on normalizing flows
• Sander on typicality
• Sander on waveform audio
• yuge shi’s ELBO gradient post is excellent
• Francis Bach, the many faces of integration by parts.
• Efficiently sampling functions from Gaussian process posteriors
• Temenin on Riemannian GPs
• https://jaan.io/what-is-variational-autoencoder-vae-tutorial/
• Whittle likelihood
• “Sethuraman” rep of Dirichlet proc = stick breaking
• Bubeck on hot results in learning theory takes him far from the world of mirror descent. Also lectures well, IMO.
• Causality for Machine Learning

learning SDEs

• [1912.05509] The Wasserstein-Fourier Distance for Stationary Time Series
• [2001.04385] Universal Differential Equations for Scientific Machine Learning
• 18.337J/6.338J: Parallel Computing and Scientific Machine Learning
• 2007.09660.pdf
• 2010.00746.pdf
• 978-0-387-21540-2_8.pdf
• AaltoML/SDE: Example codes for the book Applied Stochastic Differential Equations
• An Implicit/Explicit CUDA-Accelerated Solver for the 2D Beeler-Reuter Model
• DiffEqFlux.jl – A Julia Library for Neural Differential Equations
• DiffEqFlux.jl: Generalized Physics-Informed and Scientific Machine Learning (SciML) · DiffEqFlux.jl
• DifferentialEquations.jl: Scientific Machine Learning (SciML) Enabled Simulation and Estimation · DifferentialEquations.jl
• dnncode/Spatio-Temporal-Model-for-SPDE: Statistical Modeling for Spatio-Temporal Data from Physical Convection-Diffusion Processes
• Empirical Arithmetic Averaging Over the Compact Stiefel Manifold - IEEE Journals & Magazine
• Generalizing Automatic Differentiation to Automatic Sparsity, Uncertainty, Stability, and Parallelism - Stochastic Lifestyle
• GitHub - mitmath/18303 at spring19
• Global Sensitivity Analysis · DifferentialEquations.jl
• Hilbert space methods for reduced-rank Gaussian process regression
• How can the general Green’s function of a linear homogeneous differential equation be derived? - Mathematics Stack Exchange
• ICERM SciML conf
• Introduction to Scientific Machine Learning 2: Physics-Informed Neural Networks - YouTube
• Introduction to Scientific Machine Learning through Physics-Informed Neural Networks
• JB_grothendieck_proof.pdf
• JuliaSim - Julia Computing
• kar12Supple.pdf
• Kolmogorov Backward Equations
• mitmath/18303: 18.303 - Linear PDEs course
• ModelingToolkit, Modelica, and Modia: The Composable Modeling Future in Julia - Stochastic Lifestyle
• Neural SDEs: Deep Generative Models in the Diffusion Limit - Maxim Raginsky - YouTube
• NeuralNetDiffEq.jl: A Neural Network solver for ODEs
• Noise Processes · DifferentialEquations.jl
• Parameter Estimation and Bayesian Analysis · DifferentialEquations.jl
• Physics-Informed Neural Networks solver · NeuralPDE.jl
• SciML: Open Source Software for Scientific Machine Learning* random_fields.pdf
• SciML/DiffEqFlux.jl: Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods
• SciML/NeuralPDE.jl: Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation
• SciML/SciMLTutorials.jl: Tutorials for doing scientific machine learning (SciML) and high-performance differential equation solving with open source software.
• Score-Based Generative Modeling through Stochastic Differential Equations
• Seminars & Workshops | DATA DRIVEN SCIENCE & ENGINEERING
• Solving Systems of Stochastic PDEs and using GPUs in Julia - Stochastic Lifestyle
• Structured Random Matrices
• Verdinelli, Wasserman : Hybrid Wasserstein distance and fast distribution clustering
• willtebbutt/TemporalGPs.jl: Fast inference for Gaussian processes in problems involving time

General emulation

Sensitivity Analysis

Spectral bizness

• QuantEcon/lecture-source-jl: Source files for “Lectures in Quantitative Economics” -- Julia version
• mschauer/Kalman.jl: Flexible filtering and smoothing in Julia
• QuantEcon.jl/kalman.jl at master · QuantEcon/QuantEcon.jl
• Whittle.pdf
• time_series - spectral_estimation.pdf
• mschauer/Kalman.jl: Flexible filtering and smoothing in Julia
• A First Look at the Kalman Filter – Quantitative Economics with Julia
• How a Kalman filter works, in pictures | Bzarg

SDEs in optimisation

To explore: Gradient flows gradient flow.

Nguyen and Malinsky (2020)

Statistical Inference via Convex Optimization.

Conjugate functions illustrated.

Francis Bach on the use of geometric sums and a different take by Julyan Arbel.

Tutorial to approximating differentiable control problems. An extension of this is universal differential equations.

PDEs

• Bringing PDEs to JAX with forward and reverse modes automatic differentiation

References