A placeholder to discuss state filtering and parameter estimation where the unobserved state is quantified by variationally-learned distributions.

Campbell et al. (2021) introduce an elegant method which also performs system identification. I would like to have time to go into more detail about this but for now I will present the key insight without adequate explanation for my own benefit. The neat trick is that the variational approximation is in a sense global, in that it all telescopes into one big variational approximation, rather than a sequence of successive approximations, each of which accumulates a greater error inside the ELBO. Intuitively this gives us more hope that we are can avoid accumulating bias at each filter step.

\[ \max _{\theta, \phi} \mathcal{L}_{t}(\theta, \phi)=\mathbb{E}_{q_{t}^{\phi}\left(x_{1: t}\right)}\left[\log \frac{p_{\theta}\left(x_{1: t}, y^{t}\right)}{q_{t}^{\phi}\left(x_{1: t}\right)}\right] \]

Our key factorization: \(q_{t}^{\phi}\left(x_{1: t}\right)=q_{t}^{\phi}\left(x_{t}\right) q_{t}^{\phi}\left(x_{t-1} \mid x_{t}\right) q_{t-1}^{\phi}\left(x_{t-2} \mid x_{t-1}\right) \ldots q_{2}^{\phi}\left(x_{1} \mid x_{2}\right)\)

True factorization: \(p_{\theta}\left(x_{t} \mid y^{t}\right) p_{\theta}\left(x_{t-1} \mid x_{t}, y^{t-1}\right) p_{\theta}\left(x_{t-2} \mid x_{t-1}, y^{t-2}\right) \cdots p_{\theta}\left(x_{1} \mid x_{2}, y^{1}\right)\)

## References

*arXiv:1511.07367 [Stat]*, November.

*Quarterly Journal of the Royal Meteorological Society*143 (703): 607β33.

*arXiv:1411.7610 [Cs, Stat]*, November.

*Advances in Neural Information Processing Systems 28*, edited by C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, 2980β88. Curran Associates, Inc.

*International Journal of Approximate Reasoning*104 (January): 185β204.

*Advances in Neural Information Processing Systems 24*, edited by J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, 2510β18. Curran Associates, Inc.

*arXiv:1801.10395 [Stat]*, January.

*Bayesian Analysis*11 (2): 325β52.

*Advances in Neural Information Processing Systems 30*, edited by I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, 5309β19. Curran Associates, Inc.

*Proceedings of ICLR*.

*arXiv:1711.00799 [Stat]*, November.

*arXiv:1704.02798 [Cs, Stat]*, April.

*Advances in Neural Information Processing Systems 29*, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 2199β2207. Curran Associates, Inc.

*Cambridge University Engineering Department, Cambridge, England, Technical Report TR-328*.

*Advances in Neural Information Processing Systems 27*, edited by Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, 3680β88. Curran Associates, Inc.

*Advances in Neural Information Processing Systems 26*, edited by C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, 3156β64. Curran Associates, Inc.

*NeuroImage*41 (3): 747β66.

*Advances in Neural Information Processing Systems 28*, edited by C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, 2629β37. Curran Associates, Inc.

*arXiv:1206.7051 [Cs, Stat]*14 (1).

*arXiv:1709.07902 [Cs, Eess, Stat]*.

*Proceedings of ICLR*.

*Autonomous Robots*, 27:75β90.

*Mathematical and Computer Modelling of Dynamical Systems*11 (4): 411β24.

*arXiv Preprint arXiv:1511.05121*.

*Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence*, 2101β9.

*Proceedings of The 25th International Conference on Artificial Intelligence and Statistics*, 875β95. PMLR.

*arXiv Preprint arXiv:1705.10306*.

*Signal Analysis and Prediction*, 163β73. Applied and Numerical Harmonic Analysis. BirkhΓ€user, Boston, MA.

*Proceedings of the IEEE*95 (6): 1295β1322.

*arXiv Preprint arXiv:1603.04733*, 1708β16.

*arXiv Preprint arXiv:1705.09279*.

*Journal of Process Control*, DYCOPS-CAB 2016, 60 (December): 82β94.

*Proceedings of ICLR*.

*arXiv Preprint arXiv:1705.11140*.

*Advances in Neural Information Processing Systems 29*, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 496β504. Curran Associates, Inc.

*PMLR*, 324β33.

*arXiv:1802.03335 [Stat]*, February.

*2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP)*, 1β6.

*IEEE Transactions on Automatic Control*54 (3): 596β600.

*arXiv:2103.10153 [Cs, Stat]*, June.

*Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics*, 844β51.

*Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics*, 868β75.

## No comments yet. Why not leave one?