Statistics of spatio-temporal processes

September 11, 2020 — September 11, 2020

Gaussian

Hilbert space

kernel tricks

spatial

statistics

stochastic processes

time series

The dynamics of spatial processes evolving in time. This notebook covers the abstract case; many more specific examples can be found in oceanography, atmospheric science, computational fluid dynamics…

Clearly, there are many different problems one might wonder about here. I am thinking in particular of the kind of problem whose discretisation might look like this, as a graphical model.

This is highly stylized — I’ve imagined there is one spatial dimension, but usually there would be two or three. The observed notes are where we have sensors that can measure the state of some parameter of interest \(w\) which evolves in time \(t\). I am wondering what we need to control for to simultaneously learn the parameters of the spatial field \(r_i\), the (possibly emulated) process \(p\) and the state of the unobserved \(w\) nodes.

1 Intros

Cosma Shalizi’s Data Over Space and Time course.

2 Ensemble Kalman Filters

A classic; happens to work pretty well with spatial fields. See Ensemble Kalman Filters.

3 Laplace approximation in spatial fields

AFAICT usually the justification we use for applying Gaussian process formalism to inference. See Laplace approximation for a background.

4 Low rank spatial fields

Fixed-Rank Kriging etc.

5 Tools

See python spatial and R spatial software.

6 References

Alzraiee, White, Knowling, et al. 2022. “A Scalable Model-Independent Iterative Data Assimilation Tool for Sequential and Batch Estimation of High Dimensional Model Parameters and States.” Environmental Modelling & Software.

Ayed, and de Bézenac. 2019. “Learning Dynamical Systems from Partial Observations.” In Advances In Neural Information Processing Systems.

Bakka, Rue, Fuglstad, et al. 2018. “Spatial Modeling with R-INLA: A Review.” WIREs Computational Statistics.

Banerjee, Gelfand, Finley, et al. 2008. “Gaussian Predictive Process Models for Large Spatial Data Sets.” Journal of the Royal Statistical Society: Series B (Statistical Methodology).

Bertolacci. 2019. “Hierarchical Bayesian Mixture Models for Spatiotemporal Data with Nonstandard Features.”

Brix, and Diggle. 2001. “Spatiotemporal Prediction for Log-Gaussian Cox Processes.” Journal of the Royal Statistical Society: Series B (Statistical Methodology).

Chen, Genton, and Sun. 2021. “Space-Time Covariance Structures and Models.” Annual Review of Statistics and Its Application.

Cressie, and Huang. 1999. “Classes of Nonseparable, Spatio-Temporal Stationary Covariance Functions.” Journal of the American Statistical Association.

Cressie, Sainsbury-Dale, and Zammit-Mangion. 2022. “Basis-Function Models in Spatial Statistics.” Annual Review of Statistics and Its Application.

Cressie, Shi, and Kang. 2010. “Fixed Rank Filtering for Spatio-Temporal Data.” Journal of Computational and Graphical Statistics.

Cressie, and Wikle. 2011. Statistics for Spatio-Temporal Data. Wiley Series in Probability and Statistics 2.0.

———. 2014. “Space-Time Kalman Filter.” In Wiley StatsRef: Statistics Reference Online.

Curtain. 1975. “Infinite-Dimensional Filtering.” SIAM Journal on Control.

Díaz-Avalos, Juan, and Mateu. 2012. “Similarity Measures of Conditional Intensity Functions to Test Separability in Multidimensional Point Processes.” Stochastic Environmental Research and Risk Assessment.

Dubrule. 2018. “Kriging, Splines, Conditional Simulation, Bayesian Inversion and Ensemble Kalman Filtering.” In Handbook of Mathematical Geosciences: Fifty Years of IAMG.

Duffin, Cripps, Stemler, et al. 2021. “Statistical Finite Elements for Misspecified Models.” Proceedings of the National Academy of Sciences.

Filippi, Mallet, and Nader. 2014. “Representation and Evaluation of Wildfire Propagation Simulations.” International Journal of Wildland Fire.

Finkenstädt, Held, and Isham. 2007. Statistical methods for spatio-temporal systems.

Forbes, Cook, and Hyndman. 2020. “Spatial Modelling of the Two-Party Preferred Vote in Australian Federal Elections: 2001–2016.” Australian & New Zealand Journal of Statistics.

Giannakis, and Das. 2017. “Extraction and Prediction of Coherent Patterns in Incompressible Flows Through Space-Time Koopman Analysis.” arXiv:1706.06450 [Math].

Giannakis, Ourmazd, Slawinska, et al. 2017. “Spatiotemporal Pattern Extraction by Spectral Analysis of Vector-Valued Observables.” arXiv:1711.02798 [Physics].

Giannakis, Slawinska, Ourmazd, et al. 2018. “Vector-Valued Spectral Analysis of Space-Time Data.” arXiv:1805.09134 [Physics].

Gneiting. 2002. “Nonseparable, Stationary Covariance Functions for Space–Time Data.” Journal of the American Statistical Association.

Godfried, Mahajan, Wang, et al. 2020. “FlowDB a Large Scale Precipitation, River, and Flash Flood Dataset.” arXiv:2012.11154 [Cs].

Gopalan, and Wikle. 2021. “A Higher-Order Singular Value Decomposition Tensor Emulator for Spatiotemporal Simulators.” Journal of Agricultural, Biological and Environmental Statistics.

Goroshin, Bruna, Tompson, et al. 2014. “Unsupervised Learning of Spatiotemporally Coherent Metrics.” arXiv:1412.6056 [Cs].

Haran. 2011. “Gaussian Random Field Models for Spatial Data.” In Handbook of Markov Chain Monte Carlo.

Higdon. 2002. “Space and Space-Time Modeling Using Process Convolutions.” In Quantitative Methods for Current Environmental Issues.

Hildeman, Bolin, and Rychlik. 2019. “Joint Spatial Modeling of Significant Wave Height and Wave Period Using the SPDE Approach.” arXiv:1906.00286 [Stat].

Hoffimann, Zortea, de Carvalho, et al. 2021. “Geostatistical Learning: Challenges and Opportunities.” Frontiers in Applied Mathematics and Statistics.

Hooten, and Wikle. 2008. “A Hierarchical Bayesian Non-Linear Spatio-Temporal Model for the Spread of Invasive Species with Application to the Eurasian Collared-Dove.” Environmental and Ecological Statistics.

Hu, and Steinsland. 2016. “Spatial Modeling with System of Stochastic Partial Differential Equations.” WIREs Computational Statistics.

Hu, Steinsland, Simpson, et al. 2013. “Spatial Modelling of Temperature and Humidity Using Systems of Stochastic Partial Differential Equations.”

Huston. 2015. “Thoughts on Spatio-Temporal Uncertainty Metrics Motivated by Input Sensitivity in the Spark Bushfire Spread Model.” In Weber, T., McPhee, M.J. And Anderssen, R.S. (Eds) MODSIM2015, 21st International Congress on Modelling and Simulation.

Iranzad, Liu, Chaovalitwongse, et al. 2021. “Boost-S: Gradient Boosted Trees for Spatial Data and Its Application to FDG-PET Imaging Data.”

Koner, and Staicu. 2023. “Second-Generation Functional Data.” Annual Review of Statistics and Its Application.

Lienen, and Günnemann. 2021. “Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networks.” In International Conference on Learning Representations.

Lindgren, and Rue. 2015. “Bayesian Spatial Modelling with R-INLA.” Journal of Statistical Software.

Liu, Yeo, and Lu. 2020. “Statistical Modeling for Spatio-Temporal Data From Stochastic Convection-Diffusion Processes.” Journal of the American Statistical Association.

Nguyen, Brandstetter, Kapoor, et al. 2023. “ClimaX: A Foundation Model for Weather and Climate.”

Nikitin, John, Solin, et al. 2022. “Non-Separable Spatio-Temporal Graph Kernels via SPDEs.” arXiv:2111.08524 [Cs, Stat].

Park, Yoo, and Nadiga. 2019. “Machine Learning Climate Variability.” In.

Patraucean, Handa, and Cipolla. 2015. “Spatio-Temporal Video Autoencoder with Differentiable Memory.” arXiv:1511.06309 [Cs].

Patterson, Levin, Staver, et al. 2020. “Probabilistic Foundations of Spatial Mean-Field Models in Ecology and Applications.” SIAM Journal on Applied Dynamical Systems.

Pewsey, and García-Portugués. 2020. “Recent Advances in Directional Statistics.” arXiv:2005.06889 [Stat].

Rackauckas, Chris, Edelman, Fischer, et al. 2020. “Generalized Physics-Informed Learning Through Language-Wide Differentiable Programming.” MIT Web Domain.

Rackauckas, Christopher, Ma, Dixit, et al. 2018. “A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions.” arXiv:1812.01892 [Cs].

Rackauckas, Christopher, Ma, Martensen, et al. 2020. “Universal Differential Equations for Scientific Machine Learning.” arXiv.org.

Raissi, Perdikaris, and Karniadakis. 2019. “Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations.” Journal of Computational Physics.

Rathbun. 1996. “Asymptotic Properties of the Maximum Likelihood Estimator for Spatio-Temporal Point Processes.” Journal of Statistical Planning and Inference.

Reich. 2012. “Spatiotemporal Quantile Regression for Detecting Distributional Changes in Environmental Processes.” Journal of the Royal Statistical Society: Series C (Applied Statistics).

Reich, Fuentes, and Dunson. 2011. “Bayesian Spatial Quantile Regression.” Journal of the American Statistical Association.

Santos-Fernandez, Hoef, Peterson, et al. 2021. “Bayesian Spatio-Temporal Models for Stream Networks.”

Särkkä. 2011. “Linear Operators and Stochastic Partial Differential Equations in Gaussian Process Regression.” In Artificial Neural Networks and Machine Learning – ICANN 2011. Lecture Notes in Computer Science.

Särkkä, and Hartikainen. 2012. “Infinite-Dimensional Kalman Filtering Approach to Spatio-Temporal Gaussian Process Regression.” In Artificial Intelligence and Statistics.

Särkkä, and Solin. 2019. Applied Stochastic Differential Equations. Institute of Mathematical Statistics Textbooks 10.

Särkkä, Solin, and Hartikainen. 2013. “Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing: A Look at Gaussian Process Regression Through Kalman Filtering.” IEEE Signal Processing Magazine.

Scheider, Gräler, Pebesma, et al. 2016. “Modeling Spatiotemporal Information Generation.” International Journal of Geographical Information Science.

Shankar, Portwood, Mohan, et al. 2020. “Learning Non-Linear Spatio-Temporal Dynamics with Convolutional Neural ODEs.” In Third Workshop on Machine Learning and the Physical Sciences (NeurIPS 2020).

Sigrist, Fabio Roman Albert. 2013. “Physics Based Dynamic Modeling of Space-Time Data.” Application/pdf.

Sigrist, Fabio, Künsch, and Stahel. 2015a. “Spate : An R Package for Spatio-Temporal Modeling with a Stochastic Advection-Diffusion Process.” Application/pdf. Journal of Statistical Software.

———. 2015b. “Stochastic Partial Differential Equation Based Modelling of Large Space-Time Data Sets.” Journal of the Royal Statistical Society: Series B (Statistical Methodology).

Solin. 2016. “Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression.”

Solin, and Särkkä. 2013. “Infinite-Dimensional Bayesian Filtering for Detection of Quasiperiodic Phenomena in Spatiotemporal Data.” Physical Review E.

Szehr, Azzimonti, and Azzimonti. 2020. “An Exact Kernel Framework for Spatio-Temporal Dynamics.” arXiv:2011.06848 [Math, Stat].

Tait, and Damoulas. 2020. “Variational Autoencoding of PDE Inverse Problems.” arXiv:2006.15641 [Cs, Stat].

Tsyrulnikov, and Rakitko. 2019. “Impact of Non-Stationarity on Hybrid Ensemble Filters: A Study with a Doubly Stochastic Advection-Diffusion-Decay Model.” Quarterly Journal of the Royal Meteorological Society.

Wikle, Berliner, and Cressie. 1998. “Hierarchical Bayesian Space-Time Models.” Environmental and Ecological Statistics.

Wikle, Cressie, and Zammit-Mangion. 2019. Spatio-Temporal Statistics with R.

Wikle, and Hooten. 2010. “A General Science-Based Framework for Dynamical Spatio-Temporal Models.” TEST.

Wikle, and Zammit-Mangion. 2023. “Statistical Deep Learning for Spatial and Spatiotemporal Data.” Annual Review of Statistics and Its Application.

Zammit-Mangion, and Wikle. 2020. “Deep Integro-Difference Equation Models for Spatio-Temporal Forecasting.” Spatial Statistics.

Zhong, Ribeiro Amaral, and Moraga. 2024. “Spatial Data Fusion Adjusting for Preferential Sampling Using Integrated Nested Laplace Approximation and Stochastic Partial Differential Equation.” Journal of the Royal Statistical Society Series A: Statistics in Society.

Zhu, and Fan. 2022. “A Synthetic Likelihood Approach for Intractable Markov Random Fields.” Computational Statistics.