Steinβs method meets variational inference via kernels and probability measures. The result is method of inference which maintains an ensemble of particles which notionally collectively sample from some target distribution. I should learn about this, as one of the methods I might use for low-assumption Bayes inference.

Let us examine the computable *kernelized* Stein discrepancy, invented in Q. Liu, Lee, and Jordan (2016), weaponized in Q. Liu, Lee, and Jordan (2016) and summarised in Xu and Matsuda (2021):

Let \(q\) be a smooth probability density on \(\mathbb{R}^{d} .\) For a smooth function \(\mathbf{f}=\) \(\left(f_{1}, \ldots, f_{d}\right): \mathbb{R}^{d} \rightarrow \mathbb{R}^{d}\), the Stein operator \(\mathcal{T}_{q}\) is defined by \[ \mathcal{T}_{q} \mathbf{f}(x)=\sum_{i=1}^{d}\left(f_{i}(x) \frac{\partial}{\partial x^{i}} \log q(x)+\frac{\partial}{\partial x^{i}} f_{i}(x)\right) \]

β¦Let \(\mathcal{H}\) be a reproducing kernel Hilbert space \((\mathrm{RKHS})\) on \(\mathbb{R}^{d}\) and \(\mathcal{H}^{d}\) be its product. By using Stein operator, kernel Stein discrepancy (KSD) (Gorham and Mackey 2015; Ley, Reinert, and Swan 2017) between two densities \(p\) and \(q\) is defined as \[ \operatorname{KSD}(p \| q)=\sup _{\|\mathbf{f}\|_{\mathcal{H}} \leq 1} \mathbb{E}_{p}\left[\mathcal{T}_{q} \mathbf{f}\right] \] It is shown that \(\operatorname{KSD}(p \| q) \geq 0\) and \(\mathrm{KSD}(p \| q)=0\) if and only if \(p=q\) under mild regularity conditions (Chwialkowski, Strathmann, and Gretton 2016). Thus, KSD is a proper discrepancy measure between densities. After some calculation, \(\operatorname{KSD}(p \| q)\) is rewritten as \[ \operatorname{KSD}^{2}(p \| q)=\mathbb{E}_{x, \tilde{x} \sim p}\left[h_{q}(x, \tilde{x})\right] \] where \(h_{q}\) does not involve \(p\).

TBD.

## Moment matching interpretation

## Incoming

## References

*Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference*, 93β117. PMLR.

*Proceedings of the 32Nd International Conference on Neural Information Processing Systems*, 2478β87. NIPSβ18. USA: Curran Associates Inc.

*Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48*, 2606β15. ICMLβ16. New York, NY, USA: JMLR.org.

*Proceedings of the 32nd International Conference on Neural Information Processing Systems*, 9187β97. NIPSβ18. Red Hook, NY, USA: Curran Associates Inc.

*arXiv:1901.07987 [Cs, Stat]*, May.

*UAI 2017*. arXiv.

*Proceedings of the 36th International Conference on Machine Learning*, 2347β56. PMLR.

*Advances in Neural Information Processing Systems*. Vol. 28.

*arXiv:2007.02857 [Cs, Math, Stat]*, October.

*Proceedings of the 35th International Conference on Machine Learning*, 1900β1908. PMLR.

*arXiv:1806.10234 [Cs, Stat]*, June.

*Probability Surveys*14 (none): 1β52.

*Proceedings of the AAAI Conference on Artificial Intelligence*32 (1).

*Proceedings of The 33rd International Conference on Machine Learning*, 9.

*Proceedings of the 32nd International Conference on Neural Information Processing Systems*, 31:8868β77. NIPSβ18. Red Hook, NY, USA: Curran Associates Inc.

*Advances In Neural Information Processing Systems*.

*Proceedings of The 25th International Conference on Artificial Intelligence and Statistics*, 2002β21. PMLR.

*Journal of Computational Physics*396 (November): 400β415.

*Computational Science β ICCS 2019. ICCS 2019. Lecture Notes in Computer Science*, edited by Joao M. F. Rodrigues, Pedro J. S. Cardoso, Janio Monteiro, Roberto Lam, Valeria V. Krzhizhanovskaya, Michael H. Lees, Jack J. Dongarra, and Peter M. A. Sloot, 141β55. Faro, Portugal: Springer.

*Mathematical Geosciences*53 (3): 375β93.

*Nonlinear Processes in Geophysics*28 (3): 295β309.

*Proceedings of the 36th International Conference on Machine Learning*, 6576β85. PMLR.

*Proceedings of the 33rd International Conference on Neural Information Processing Systems*, 7836β46. Red Hook, NY, USA: Curran Associates Inc.

*Statistics and Computing*32 (6): 97.

*arXiv:2103.00895 [Stat]*, March.

*International Conference on Artificial Intelligence and Statistics*, 1877β87. PMLR.

*Proceedings of the 35th International Conference on Machine Learning*, 6018β27. PMLR.

## No comments yet. Why not leave one?