# Particle filters

incorporating Interacting Particle Systems, Sequential Monte Carlo and a profusion of other simultaneous-discovery names

July 25, 2014 — March 24, 2023

A Monte Carlo algorithm which updates a population of samples with a nested update. The easiest entry point is IMO to think about random-sample generalisation of state filter models via importance sampling. These are classically considered cousins to the linear Gaussian Kalman filter, applicable to more challenging models at the cost of using Monte Carlo approximations.

This has nothing to do with filters for particulate matter as seen in respirators.

There is too much confusing and unhelpful terminology here, and I am only at the fringe of this field so I will not attempt to typologize. Let us clear up the main stumbling block though: somehow the theoretical basis field has coalesced under the banner of *interacting particle systems* which is an awful unsearchable name which could mean anything, and indeed does in other disciplines. Wikipedia disambiguates this problem with the gritty and also abstruse Mean Field Particle Methods. In practical applications we talk about *particle filters*, or *sequential Monte Carlo*, or *bootstrap filters*, or *iterated importance sampling* and these all mean confusingly similar things.

## 1 Introductions

- Pierre Jacob’s Particle methods for statistics reading list
- The lineage and reasoning is well explained by Cappe, Godsill, and Moulines (2007).
- Chopin and Papaspiliopoulos (2020)

Easy to explain with an example such as this particle filter in scala.

## 2 Feynman-Kac formulae

See Feynman-Kac.

## 3 System Identification in

Do not know the parameters governing the system dynamics and need to learn those too? See System identification with particle fitlers.

## 4 Relation to Ensemble Kalman filters

Yes.

## 5 Particle flow

Introduced to me by my colleague Adrian Bishop Profile.

Bunch and Godsill (2014); Daum, Huang, and Noushin (2010); Daum and Huang (2010); Daum and Huang (2009); Daum and Huang (2008); Daum and Huang (2013)

## 6 Non-Gaussian evolution

### 6.1 Jump process

I am interested in jump-process SMC (to be defined). For those I should apparently consult Graham and Méléard (1997);Grünbaum (1971);Méléard (1996);Shiga and Tanaka (1985).

### 6.2 On weird graphs

Christian Andersson Naesseth, Lindsten, and Schön (2014)

## 7 Rao-Blackwellized particle filter

Particles which represent marginalised densities (Murphy 2012, 23.6).

## 8 Tooling

Some MCMC toolkits incorporate SMC too.

- particles is a python library for teaching DIY particle filtering, to accompany the book Chopin and Papaspiliopoulos (2020).
- Johansen’s page, with C++ software
- Dirk Eddelbuettel’s lab created RCppSMC for R integration of the Johansen stuff. Documentation is not great — it only consists of black-box toy problems without any hint of how you would construct, e.g. a likelihood function, so I can’t evaluate how easy this would be to use, as opp plain C++.
- most probabilistic programming languages include a particle filter example.

## 9 References

*Bernoulli*.

*Applied Sciences*.

*Scientific Reports*.

*Handbook of Econometrics*.

*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*.

*IEEE Transactions on Signal Processing*.

*Nature Reviews Methods Primers*.

*The Annals of Applied Statistics*.

*Proceedings of the IEEE*.

*The Annals of Applied Probability*.

*Statistics and Computing*.

*Econometric Theory*.

*An Introduction to Sequential Monte Carlo*. Springer Series in Statistics.

*arXiv:2102.07850 [Cs, Stat]*.

*Markov Processes and Related Fields*.

*Bernoulli*.

*Signal and Data Processing of Small Targets 2008*.

*Signal and Data Processing of Small Targets 2009*.

*Signal and Data Processing of Small Targets 2010*.

*Signal Processing, Sensor Fusion, and Target Recognition XXII*.

*Signal Processing, Sensor Fusion, and Target Recognition XIX*.

*Journal of Econometrics*, Semiparametric methods in econometrics,.

*Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications*.

*The Annals of Applied Probability*.

*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*.

*Statistics and Computing*.

*On the Concentration Properties of Interacting Particle Processes*.

*Stochastic Hybrid Systems*. Lecture Notes in Control and Information Science, Volume 337.

*Séminaire de Probabilités XXXIV*. Lecture Notes in Mathematics 1729.

*Sequential Monte Carlo Methods in Practice*.

*Sequential Monte Carlo Methods in Practice*. Statistics for Engineering and Information Science.

*Statistics and Computing*.

*arXiv:1304.5768 [Stat]*.

*Handbook of Nonlinear Filtering*.

*Bayesian Analysis*.

*Journal of Statistical Software*.

*Data Assimilation - The Ensemble Kalman Filter*.

*Annual Review of Statistics and Its Application*.

*SIAM Journal on Applied Dynamical Systems*.

*Mathematics of Control, Signals and Systems*.

*The Annals of Probability*.

*Archive for Rational Mechanics and Analysis*.

*Advances in Neural Information Processing Systems 28*.

*Review of Financial Studies*.

*IEEE Transactions on Signal Processing*.

*The Annals of Statistics*.

*Journal of Statistical Software*.

*Proceedings of the 6th International Workshop on Rare Event Simulation*.

*arXiv:1805.11122 [Cs, Stat]*.

*Statistical Science*.

*Journal of Statistical Physics*.

*The American Statistician*.

*Computer Vision – ACCV 2007*. Lecture Notes in Computer Science 4843.

*The Annals of Statistics*.

*Bernoulli*.

*Proceedings of the 34th International Conference on Neural Information Processing Systems*. NIPS ’20.

*Proceedings of The 25th International Conference on Artificial Intelligence and Statistics*.

*arXiv:1509.00394 [Stat]*.

*Discrete & Continuous Dynamical Systems - A*.

*Advances In Neural Information Processing Systems*.

*Proceedings of the 36th International Conference on Machine Learning*.

*arXiv Preprint arXiv:1705.09279*.

*Probabilistic Models for Nonlinear Partial Differential Equations: Lectures Given at the 1st Session of the Centro Internazionale Matematico Estivo (C.I.M.E.) Held in Montecatini Terme, Italy, May 22–30, 1995*. Lecture Notes in Mathematics.

*Machine learning: a probabilistic perspective*. Adaptive computation and machine learning series.

*Advances in Neural Information Processing Systems*.

*arXiv:1903.04797 [Cs, Stat]*.

*Conference Record of the Thirty-Eighth Asilomar Conference on Signals, Systems and Computers, 2004*.

*Acta Numerica*.

*arXiv:1408.4908 [Cs, Math, q-Bio, Stat]*.

*Biometrika*.

*Journal of Time Series Analysis*.

*Simulation and the Monte Carlo Method*. Wiley series in probability and statistics.

*Fast Sequential Monte Carlo Methods for Counting and Optimization*. Wiley Series in Probability and Statistics.

*Physical Review E*.

*Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete*.

*Proceedings of the National Academy of Sciences*.

*Proceedings of the National Academy of Sciences*.

*IEEE Transactions on Automatic Control*.

*Statistics and Computing*.

*arXiv:2007.02692 [q-Fin]*.

*Proceedings of The 27th International Conference on Artificial Intelligence and Statistics*.