# Log concave distributions

associated tools

August 28, 2017 — March 11, 2021

“a Markov Chain reminiscent of noisy gradient descent”. Holden Lee, Andrej Risteski introduce this the connection between log-concavity and convex optimisation.

\[ x_{t+\eta} = x_t - \eta \nabla f(x_t) + \sqrt{2\eta}\xi_t,\quad \xi_t\sim N(0,I). \]

## 1 Langevin MCMC

See SGD MCMC for now.

Rob Salomone explains this well; see Hodgkinson, Salomone, and Roosta (2019).

Andrej Risteski’s Beyond log-concave sampling series is a also a good introduction to log-concave sampling.

## 2 References

Bagnoli, and Bergstrom. 1989. “Log-Concave Probability and Its Applications.”

Brosse, Moulines, and Durmus. 2018. “The Promises and Pitfalls of Stochastic Gradient Langevin Dynamics.” In

*Proceedings of the 32nd International Conference on Neural Information Processing Systems*. NIPS’18.
Castellani, and Cavagna. 2005. “Spin-Glass Theory for Pedestrians.”

*Journal of Statistical Mechanics: Theory and Experiment*.
Domke. 2017. “A Divergence Bound for Hybrids of MCMC and Variational Inference and an Application to Langevin Dynamics and SGVI.” In

*PMLR*.
Duane, Kennedy, Pendleton, et al. 1987. “Hybrid Monte Carlo.”

*Physics Letters B*.
Durmus, and Moulines. 2016. “High-Dimensional Bayesian Inference via the Unadjusted Langevin Algorithm.”

*arXiv:1605.01559 [Math, Stat]*.
Garbuno-Inigo, Hoffmann, Li, et al. 2020. “Interacting Langevin Diffusions: Gradient Structure and Ensemble Kalman Sampler.”

*SIAM Journal on Applied Dynamical Systems*.
Ge, Lee, and Risteski. 2020. “Simulated Tempering Langevin Monte Carlo II: An Improved Proof Using Soft Markov Chain Decomposition.”

*arXiv:1812.00793 [Cs, Math, Stat]*.
Girolami, and Calderhead. 2011. “Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods.”

*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*.
Hodgkinson, Salomone, and Roosta. 2019. “Implicit Langevin Algorithms for Sampling From Log-Concave Densities.”

*arXiv:1903.12322 [Cs, Stat]*.
Mandt, Hoffman, and Blei. 2017. “Stochastic Gradient Descent as Approximate Bayesian Inference.”

*JMLR*.
Mangoubi, and Smith. 2017. “Rapid Mixing of Hamiltonian Monte Carlo on Strongly Log-Concave Distributions.”

*arXiv:1708.07114 [Math, Stat]*.
Norton, and Fox. 2016. “Tuning of MCMC with Langevin, Hamiltonian, and Other Stochastic Autoregressive Proposals.”

*arXiv:1610.00781 [Math, Stat]*.
Saumard, and Wellner. 2014. “Log-Concavity and Strong Log-Concavity: A Review.”

*arXiv:1404.5886 [Math, Stat]*.
Welling, and Teh. 2011. “Bayesian Learning via Stochastic Gradient Langevin Dynamics.” In

*Proceedings of the 28th International Conference on International Conference on Machine Learning*. ICML’11.
Xifara, Sherlock, Livingstone, et al. 2014. “Langevin Diffusions and the Metropolis-Adjusted Langevin Algorithm.”

*Statistics & Probability Letters*.