Change points

Looking for regime changes in stochastic processes. a.k.a. Switching state space models

November 29, 2021 — April 1, 2022

Bayes
metrics
Monte Carlo
probabilistic algorithms
signal processing
state space models
statistics
stochastic processes
time series

If our data is stationary that is great and we have good estimation theory. However, every non-stationary time series is non-stationary in its own way. A popular type of non-stationarity is change-point type where we imagine locally stationary series are stapled together.

1 Piecewise wide-sense-stationary time series

Aminikhanghahi and Cook (2017) surveys many types of change point methods, with focus on ones that use up-to-second-order statistics, i.e. piecewise wide-sense-stationary time series.

2 Online detection

Can we do online state filtering?

3 Online detection with system identification

Can we do online state filtering and identification?

4 Bayesian detection of change points

I am going to file the AdaptSpec methods in here, even though they try to marginalise over possible changepoints, because changepoints end up being fundamental to these methods and also I think they are cool . You might also imagine these to be probabilistic spectral methods.

I am typically more interested in online methods. A classic is Adams and MacKay (2007) which introduces an auxiliary run time variate which factorizes nicely. The basic algorithm only works for conditionally i.i.d. observations though. Saatçi, Turner, and Rasmussen (2010) extends it to a Gaussian process time series.

5 Functional

Where the observations are themselves functions.

6 Switching State Space models

a.k.a. mixture filters` .

7 As anomaly detection

If we have a “typical” regime with constant coefficients and an “anomalous” one without constant coefficients then we are in an anomaly detection setting.

8 References

Adams, and MacKay. 2007. arXiv:0710.3742 [Stat].
Agudelo-España, Gomez-Gonzalez, Bauer, et al. 2020. In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI).
Ahmad, Lavin, Purdy, et al. 2017. Neurocomputing, Online Real-Time Learning Strategies for Data Streams,.
Aminikhanghahi, and Cook. 2017. Knowledge and Information Systems.
Aue, Gabrys, Horváth, et al. 2009. Journal of Multivariate Analysis.
Berkes, Gabrys, Horváth, et al. 2009. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
Bertolacci, Rosen, Cripps, et al. 2020. arXiv:1908.06622 [Stat].
Detommaso, Hoitzing, Cui, et al. 2019. arXiv:1901.07987 [Cs, Stat].
———. 2005. Statistics and Computing.
———. 2011. In Handbook of Markov Chain Monte Carlo.
Fearnhead, and Clifford. 2003. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
Fearnhead, and Liu. 2007. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
———. 2011. Statistics and Computing.
Fearnhead, and Sherlock. 2006. Journal of the Royal Statistical Society: Series B (Statistical Methodology).
Gundersen, Cai, Zhou, et al. 2021. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence.
Horváth, Hušková, and Kokoszka. 2010. Journal of Multivariate Analysis, Statistical Methods and Problems in Infinite-dimensional Spaces,.
Nemeth, Fearnhead, and Mihaylova. 2014. IEEE Transactions on Signal Processing.
Rosen, Wood, and Stoffer. 2012. Journal of the American Statistical Association.
Saatçi, Turner, and Rasmussen. 2010. In Proceedings of the 27th International Conference on International Conference on Machine Learning. ICML’10.
Turner, Bottone, and Stanek. 2013. In Advances in Neural Information Processing Systems.
van Delft, Characiejus, and Dette. 2021. Statistica Sinica.
Vogt, and Dette. 2015. The Annals of Statistics.
Wilson, Nassar, and Gold. 2010. Neural Computation.