Adaptive Markov Chain Monte Carlo samplers



In adaptive MCMC, the trajectories of the simulator is perturbed by external forces (bottom right, centre) to change how they approaches the target (top right)

Designing MCMC transition density by online optimisation for optimal mixing. Also called controlled MCMC.

Here we are no longer truly using a Markov chain because the transition parameters depend upon the entire history of the chain (for example because you are dynamically updating the transition parameters to improve mixing etc). Tutorials: AtchadΓ© et al. (2011) and Andrieu and Thoms (2008).

With a Markov chain it is more complicated; If we perturb the transition density infinitely often we do not know in general that we will still converge to the target stationary distribution. However, we could do a β€œpilot” run to estimate optimal mixing kernels then use the adapted mixing kernels, discarding the samples from the pilot run as suspect and using the ones that remained. This is then a tuned MCMC rather than an adaptive MCMC.

Here I will keep notes, if any on the perturbation problem. How do we guarantee that the proposal density is not changing too much by some criterion? Solutions to this seem to be sampler-specific.

References

Andrieu, Christophe, and Christian P. Robert. 2001. β€œControlled MCMC for Optimal Sampling.” Working Paper 2001-33. Center for Research in Economics and Statistics.
Andrieu, Christophe, and Johannes Thoms. 2008. β€œA Tutorial on Adaptive MCMC.” Statistics and Computing 18 (4): 343–73.
AtchadΓ©, Yves, Gersende Fort, Eric Moulines, and Pierre Priouret. 2011. β€œAdaptive Markov Chain Monte Carlo: Theory and Methods.” In Bayesian Time Series Models, edited by David Barber, A. Taylan Cemgil, and Silvia Chiappa, 32–51. Cambridge: Cambridge University Press.
Gilks, Walter R., Gareth O. Roberts, and Sujit K. Sahu. 1998. β€œAdaptive Markov Chain Monte Carlo Through Regeneration.” Journal of the American Statistical Association 93 (443): 1045–54.
Griffin, Jim, Krys Latuszynski, and Mark Steel. 2019. β€œIn Search of Lost (Mixing) Time: Adaptive Markov Chain Monte Carlo Schemes for Bayesian Variable Selection with Very Large p.” arXiv:1708.05678 [Stat], May.
Maire, Florian, Nial Friel, Antonietta Mira, and Adrian E. Raftery. 2019. β€œAdaptive Incremental Mixture Markov Chain Monte Carlo.” Journal of Computational and Graphical Statistics 28 (4): 790–805.
Mathew, B, A M Bauer, P Koistinen, T C Reetz, J LΓ©on, and M J SillanpÀÀ. 2012. β€œBayesian Adaptive Markov Chain Monte Carlo Estimation of Genetic Parameters.” Heredity 109 (4): 235–45.
Roberts, Gareth O., and Jeffrey S. Rosenthal. 2001. β€œOptimal Scaling for Various Metropolis-Hastings Algorithms.” Statistical Science 16 (4): 351–67.
β€”β€”β€”. 2009. β€œExamples of Adaptive MCMC.” Journal of Computational and Graphical Statistics 18 (2): 349–67.
Rosenthal, Jeffrey. 2011. β€œOptimal Proposal Distributions and Adaptive MCMC.” In Handbook of Markov Chain Monte Carlo, edited by Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng. Vol. 20116022. Chapman and Hall/CRC.
Sejdinovic, Dino, Heiko Strathmann, Maria Lomeli Garcia, Christophe Andrieu, and Arthur Gretton. 2014. β€œKernel Adaptive Metropolis-Hastings.” In International Conference on Machine Learning, 1665–73. Beijing, China: JMLR.org.
Strathmann, Heiko, Dino Sejdinovic, Samuel Livingstone, Zoltan Szabo, and Arthur Gretton. 2015. β€œGradient-Free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families.” In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, 955–63. NIPS’15. Montreal, Canada: MIT Press.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.