Elliptical belief propagation

Generalized least generalized squares

August 22, 2022 — August 23, 2022

algebra
approximation
Bayes
distributed
dynamical systems
Gaussian
generative
graphical models
Hilbert space
linear algebra
machine learning
networks
optimization
probability
signal processing
state space models
statistics
stochastic processes
swarm
time series
Figure 1

We can generalize Gaussian belief propagation to use general elliptical laws by using Mahalanobis distance without presuming the Gaussian distribution (Agarwal et al. 2013; Davison and Ortiz 2019), making it into a kind of elliptical belief propagation.

1 Robust

If we use a robust Huber loss instead of a Gaussian log-likelihood, then the resulting algorithm is usually referred to as a robust factor or as dynamic covariance scaling (Agarwal et al. 2013; Davison and Ortiz 2019). The nice thing here is that we can imagine the transition from quadratic to linear losses gives us an estimate of which observations are outliers.

2 Student-\(t\)

Surely this is around? Certainly there is a special case in the t-process. It is mentioned, I think, in Lan et al. (2006) and possibly Proudler et al. (2007) although the latter seems to be something more ad hoc.

3 Gaussian mixture

Surely? TBD.

4 Generic

There seem to be generic update rules (Aste 2021; Bånkestad et al. 2020) which could be used to construct a generic elliptical belief propagation algorithm.

5 References

Agarwal, Tipaldi, Spinello, et al. 2013. Robust Map Optimization Using Dynamic Covariance Scaling.” In 2013 IEEE International Conference on Robotics and Automation.
Aste. 2021. Stress Testing and Systemic Risk Measures Using Multivariate Conditional Probability.”
Bånkestad, Sjölund, Taghia, et al. 2020. The Elliptical Processes: A Family of Fat-Tailed Stochastic Processes.”
Cambanis, Huang, and Simons. 1981. On the Theory of Elliptically Contoured Distributions.” Journal of Multivariate Analysis.
Chamberlain. 1983. A Characterization of the Distributions That Imply Mean—Variance Utility Functions.” Journal of Economic Theory.
Davison, and Ortiz. 2019. FutureMapping 2: Gaussian Belief Propagation for Spatial AI.” arXiv:1910.14139 [Cs].
Donoho, and Montanari. 2013. High Dimensional Robust M-Estimation: Asymptotic Variance via Approximate Message Passing.” arXiv:1310.7320 [Cs, Math, Stat].
Karlgaard, and Schaub. 2011. Adaptive Nonlinear Huber-Based Navigation for Rendezvous in Elliptical Orbit.” Journal of Guidance, Control, and Dynamics.
Landsman, and Nešlehová. 2008. Stein’s Lemma for Elliptical Random Vectors.” Journal of Multivariate Analysis.
Lan, Roth, Huttenlocher, et al. 2006. Efficient Belief Propagation with Learned Higher-Order Markov Random Fields.” In Computer Vision – ECCV 2006.
Ortiz, Evans, and Davison. 2021. A Visual Introduction to Gaussian Belief Propagation.” arXiv:2107.02308 [Cs].
Proudler, Roberts, Reece, et al. 2007. An Iterative Signal Detection Algorithm Based on Bayesian Belief Propagation Ideas.” In 2007 15th International Conference on Digital Signal Processing.