Anomaly detection

I don’t define what is abnormal, but I know it when I see it



An outlier in my ocean hydrology model.

Working out when something you are seeing is not something you were expecting, when you are not quite sure how to quantify what you expect actually. Detecting black swans, for example.

Nuit Blanche rounds up methods for high dimensional statistics.

Trend detection

e.g. Gnip Trend detection Nikolov (2012) looks fun, and comes with a cute explanation of how you might do this nonparametrically.

References

Ahmad, Subutai, Alexander Lavin, Scott Purdy, and Zuha Agha. 2017. β€œUnsupervised Real-Time Anomaly Detection for Streaming Data.” Neurocomputing, Online Real-Time Learning Strategies for Data Streams, 262 (November): 134–47.
Chalapathy, Raghavendra, and Sanjay Chawla. 2019. β€œDeep Learning for Anomaly Detection: A Survey.” arXiv:1901.03407 [Cs, Stat], January.
Henrickson, Scott, Jeff Kolb, Brian Lehmann, and Joshua Montague. 2015. β€œTrend Detection in Social Data.” Twitter.
Kandanaarachchi, Sevvandi. 2021. β€œUnsupervised Anomaly Detection Ensembles Using Item Response Theory.” arXiv:2106.06243 [Cs, Stat], June.
Kandanaarachchi, Sevvandi, and Rob J. Hyndman. 2021. β€œDimension Reduction for Outlier Detection Using DOBIN.” Journal of Computational and Graphical Statistics 30 (1): 204–19.
Kandanaarachchi, Sevvandi, Rob J. Hyndman, and Kate Smith-Miles. 2020. β€œEarly Classification of Spatio-Temporal Events Using Partial Information.” PLOS ONE 15 (8): e0236331.
Kandanaarachchi, Sevvandi, Mario A. Munoz, Rob J. Hyndman, and Kate Smith-Miles. 2018. β€œOn Normalization and Algorithm Selection for Unsupervised Outlier Detection.” 16/18. Monash Econometrics and Business Statistics Working Papers. Monash University, Department of Econometrics and Business Statistics.
Kulinski, Sean, and David I. Inouye. 2022. β€œTowards Explaining Distribution Shifts.” arXiv.
Lopez, Jose A., Octavia Camps, and Mario Sznaier. 2015. β€œRobust Anomaly Detection Using Semidefinite Programming.” arXiv:1504.00905 [Cs, Math], April.
Ming, Yifei, Ying Fan, and Yixuan Li. 2022. β€œPOEM: Out-of-Distribution Detection with Posterior Sampling,” June.
Nikolov, Stanislav. 2012. β€œTrend or no trend : a novel nonparametric method for classifying time series.” Thesis, Massachusetts Institute of Technology.
Ovadia, Yaniv, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, and Jasper Snoek. 2019. β€œCan You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.” In Proceedings of the 33rd International Conference on Neural Information Processing Systems, 14003–14. Red Hook, NY, USA: Curran Associates Inc.
Pang, Guansong, Chunhua Shen, Longbing Cao, and Anton van den Hengel. 2021. β€œDeep Learning for Anomaly Detection: A Review.” ACM Computing Surveys 54 (2): 1–38.
Sevvandi Kandanaarachchi, and Rob J Hyndman. 2021. β€œLeave-One-Out Kernel Density Estimates for Outlier Detection.”
She, Yiyuan, and Art B. Owen. 2010. β€œOutlier Detection Using Nonconvex Penalized Regression.”
Talagala, Priyanga Dilini, Rob J. Hyndman, Kate Smith-Miles, Sevvandi Kandanaarachchi, and Mario A. MuΓ±oz. 2020. β€œAnomaly Detection in Streaming Nonstationary Temporal Data.” Journal of Computational and Graphical Statistics 29 (1): 13–27.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.