Change points

Piecewise wide-sense-stationary time series
Online detection
Online detection with system identification
Bayesian detection of change points
Functional
Switching State Space models
As anomaly detection
References

Non-stationary dynamics turn out to be important in practice.

If our data is stationary that is great and we have good estimation theory. However, every non-stationary time series is non-stationary in its own way. A popular type of non-stationarity is change-point type where we imagine locally stationary series are stapled together.

Piecewise wide-sense-stationary time series

Aminikhanghahi and Cook (2017) surveys many types of change point methods, with focus on ones that use up-to-second-order statistics, i.e. piecewise wide-sense-stationary time series.

Online detection

Can we do online state filtering?

Online detection with system identification

Can we do online state filtering and identification?

Bayesian detection of change points

I am going to file the AdaptSpec methods in here, even though they try to marginalise over possible changepoints, because changepoints end up being fundamental to these methods and also I think they are cool (Bertolacci et al. 2020; Rosen, Wood, and Stoffer 2012). You might also imagine these to be probabilistic spectral methods.

I am typically more interested in online methods. A classic is Adams and MacKay (2007) which introduces an auxiliary run time variate which factorizes nicely. The basic algorithm only works for conditionally i.i.d. observations though. Saatçi, Turner, and Rasmussen (2010) extends it to a Gaussian process time series.

Functional

Where the observations are themselves functions.

Switching State Space models

a.k.a. mixture filters` (Fearnhead 2005, 2004; Fearnhead and Liu 2011, 2007; Fearnhead and Sherlock 2006; Fearnhead and Clifford 2003).

As anomaly detection

If we have a “typical” regime with constant coefficients and an “anomalous” one without constant coefficients then we are in an anomaly detection setting.

References

Adams, Ryan Prescott, and David J. C. MacKay. 2007. “Bayesian Online Changepoint Detection.” arXiv:0710.3742 [Stat], October.

Agudelo-España, Diego, Sebastian Gomez-Gonzalez, Stefan Bauer, Bernhard Schölkopf, and Jan Peters. 2020. “Bayesian Online Prediction of Change Points.” In Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), 320–29. PMLR.

Ahmad, Subutai, Alexander Lavin, Scott Purdy, and Zuha Agha. 2017. “Unsupervised Real-Time Anomaly Detection for Streaming Data.” Neurocomputing, Online Real-Time Learning Strategies for Data Streams, 262 (November): 134–47.

Aminikhanghahi, Samaneh, and Diane J. Cook. 2017. “A Survey of Methods for Time Series Change Point Detection.” Knowledge and Information Systems 51 (2): 339–67.

Aue, Alexander, Robertas Gabrys, Lajos Horváth, and Piotr Kokoszka. 2009. “Estimation of a Change-Point in the Mean Function of Functional Data.” Journal of Multivariate Analysis 100 (10): 2254–69.

Berkes, István, Robertas Gabrys, Lajos Horváth, and Piotr Kokoszka. 2009. “Detecting Changes in the Mean of Functional Observations.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71 (5): 927–46.

Bertolacci, Michael, Ori Rosen, Edward Cripps, and Sally Cripps. 2020. “AdaptSPEC-X: Covariate Dependent Spectral Modeling of Multiple Nonstationary Time Series.” arXiv:1908.06622 [Stat], June.

Delft, Anne van, Vaidotas Characiejus, and Holger Dette. 2021. “A Nonparametric Test for Stationarity in Functional Time Series.” Statistica Sinica.

Detommaso, Gianluca, Hanne Hoitzing, Tiangang Cui, and Ardavan Alamir. 2019. “Stein Variational Online Changepoint Detection with Applications to Hawkes Processes and Neural Networks.” arXiv:1901.07987 [Cs, Stat], May.

Fearnhead, Paul. 2004. “Particle Filters for Mixture Models with an Unknown Number of Components.” Statistics and Computing 14 (1): 11–21.

———. 2005. “Direct Simulation for Discrete Mixture Distributions.” Statistics and Computing 15 (2): 125–33.

———. 2011. “MCMC for State–Space Models.” In Handbook of Markov Chain Monte Carlo, edited by Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng. Vol. 20116022. Chapman and Hall/CRC.

Fearnhead, Paul, and Peter Clifford. 2003. “On-Line Inference for Hidden Markov Models via Particle Filters.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 65 (4): 887–99.

Fearnhead, Paul, and Zhen Liu. 2007. “On-Line Inference for Multiple Changepoint Problems.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 69 (4): 589–605.

———. 2011. “Efficient Bayesian Analysis of Multiple Changepoint Models with Dependence Across Segments.” Statistics and Computing 21 (2): 217–29.

Fearnhead, Paul, and Chris Sherlock. 2006. “An Exact Gibbs Sampler for the Markov-Modulated Poisson Process.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68 (5): 767–84.

Gundersen, Gregory W., Diana Cai, Chuteng Zhou, Barbara E. Engelhardt, and Ryan P. Adams. 2021. “Active Multi-Fidelity Bayesian Online Changepoint Detection.” In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, 1916–26. PMLR.

Horváth, Lajos, Marie Hušková, and Piotr Kokoszka. 2010. “Testing the Stability of the Functional Autoregressive Process.” Journal of Multivariate Analysis, Statistical Methods and Problems in Infinite-dimensional Spaces, 101 (2): 352–67.

Nemeth, Christopher, Paul Fearnhead, and Lyudmila Mihaylova. 2014. “Sequential Monte Carlo Methods for State and Parameter Estimation in Abruptly Changing Environments.” IEEE Transactions on Signal Processing 62 (5): 1245–55.

Rosen, Ori, Sally Wood, and David S. Stoffer. 2012. “AdaptSPEC: Adaptive Spectral Estimation for Nonstationary Time Series.” Journal of the American Statistical Association 107 (500): 1575–89.

Saatçi, Yunus, Ryan Turner, and Carl Edward Rasmussen. 2010. “Gaussian Process Change Point Models.” In Proceedings of the 27th International Conference on International Conference on Machine Learning, 927–34. ICML’10. Madison, WI, USA: Omnipress.

Turner, Ryan D, Steven Bottone, and Clay J Stanek. 2013. “Online Variational Approximations to Non-Exponential Family Change Point Models: With Application to Radar Tracking.” In Advances in Neural Information Processing Systems, 26:9. Curran Associates, Inc.

Vogt, Michael, and Holger Dette. 2015. “Detecting Gradual Changes in Locally Stationary Processes.” The Annals of Statistics 43 (2): 713–40.

Wilson, Robert C., Matthew R. Nassar, and Joshua I. Gold. 2010. “Bayesian Online Learning of the Hazard Rate in Change-Point Problems.” Neural Computation 22 (9): 2452–76.

Looking for regime changes in stochastic processes. a.k.a. Switching state space models