Neural learning for spatiotemporal systems

The known problems with out-of-sample prediction in neural nets have strong history in spatial inference generally.

On learning stochastic partial differential equations and other processes using neural networks and other differentiable techniques. Uses the tools of dynamical NNs and their ilk. Probably handy for machine learning physics and especially PDEs.

In the inverse setting any plausible inference algorithm will likely involve low rank or lattice gaussian processes.

I know little about this topic yet. But here are some articles to read.


Arridge, Simon, Peter Maass, Ozan Γ–ktem, and Carola-Bibiane SchΓΆnlieb. 2019. β€œSolving Inverse Problems Using Data-Driven Models.” Acta Numerica 28 (May): 1–174.
Ayed, Ibrahim, and Emmanuel de BΓ©zenac. 2019. β€œLearning Dynamical Systems from Partial Observations.” In Advances In Neural Information Processing Systems, 12.
Beck, Christian, Weinan E, and Arnulf Jentzen. 2019. β€œMachine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-Order Backward Stochastic Differential Equations.” Journal of Nonlinear Science 29 (4): 1563–1619.
Dupont, Emilien, Hyunjik Kim, S. M. Ali Eslami, Danilo Rezende, and Dan Rosenbaum. 2022. β€œFrom Data to Functa: Your Data Point Is a Function and You Can Treat It Like One.” arXiv.
Han, Jiequn, Arnulf Jentzen, and Weinan E. 2018. β€œSolving High-Dimensional Partial Differential Equations Using Deep Learning.” Proceedings of the National Academy of Sciences 115 (34): 8505–10.
He, QiZhi, David Barajas-Solano, Guzel Tartakovsky, and Alexandre M. Tartakovsky. 2020. β€œPhysics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport.” Advances in Water Resources 141 (July): 103610.
Laloy, Eric, and Diederik Jacques. 2019. β€œEmulation of CPU-Demanding Reactive Transport Models: A Comparison of Gaussian Processes, Polynomial Chaos Expansion, and Deep Neural Networks.” Computational Geosciences 23 (5): 1193–1215.
Lu, Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. 2021. β€œDeepXDE: A Deep Learning Library for Solving Differential Equations.” SIAM Review 63 (1): 208–28.
Mo, Shaoxing, Nicholas Zabaras, Xiaoqing Shi, and Jichun Wu. 2019. β€œDeep Autoregressive Neural Networks for High-Dimensional Inverse Problems in Groundwater Contaminant Source Identification.” Water Resources Research 55 (5): 3856–81.
Nabian, Mohammad Amin, and Hadi Meidani. 2019. β€œA Deep Learning Solution Approach for High-Dimensional Random Differential Equations.” Probabilistic Engineering Mechanics 57 (July): 14–25.
Park, Ji Hwan, Shinjae Yoo, and Balu Nadiga. 2019. β€œMachine Learning Climate Variability.” In, 5.
Patraucean, Viorica, Ankur Handa, and Roberto Cipolla. 2015. β€œSpatio-Temporal Video Autoencoder with Differentiable Memory.” arXiv:1511.06309 [Cs], November.
Rackauckas, Chris, Alan Edelman, Keno Fischer, Mike Innes, Elliot Saba, Viral B Shah, and Will Tebbutt. 2020. β€œGeneralized Physics-Informed Learning Through Language-Wide Differentiable Programming.” MIT Web Domain, 6.
Raissi, Maziar, P. Perdikaris, and George Em Karniadakis. 2019. β€œPhysics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations.” Journal of Computational Physics 378 (February): 686–707.
Raissi, Maziar, Alireza Yazdani, and George Em Karniadakis. 2020. β€œHidden Fluid Mechanics: Learning Velocity and Pressure Fields from Flow Visualizations.” Science 367 (6481): 1026–30.
Ruthotto, Lars, and Eldad Haber. 2018. β€œDeep Neural Networks Motivated by Partial Differential Equations.” arXiv:1804.04272 [Cs, Math, Stat], April.
SΓ€rkkΓ€, Simo. 2011. β€œLinear Operators and Stochastic Partial Differential Equations in Gaussian Process Regression.” In Artificial Neural Networks and Machine Learning – ICANN 2011, edited by Timo Honkela, WΕ‚odzisΕ‚aw Duch, Mark Girolami, and Samuel Kaski, 6792:151–58. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer.
SΓ€rkkΓ€, Simo, and Jouni Hartikainen. 2012. β€œInfinite-Dimensional Kalman Filtering Approach to Spatio-Temporal Gaussian Process Regression.” In Artificial Intelligence and Statistics.
SΓ€rkkΓ€, Simo, A. Solin, and J. Hartikainen. 2013. β€œSpatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing: A Look at Gaussian Process Regression Through Kalman Filtering.” IEEE Signal Processing Magazine 30 (4): 51–61.
Shankar, Varun, Gavin D Portwood, Arvind T Mohan, Peetak P Mitra, Christopher Rackauckas, Lucas A Wilson, David P Schmidt, and Venkatasubramanian Viswanathan. 2020. β€œLearning Non-Linear Spatio-Temporal Dynamics with Convolutional Neural ODEs.” In Third Workshop on Machine Learning and the Physical Sciences (NeurIPS 2020).
Solin, Arno, and Simo SΓ€rkkΓ€. 2013. β€œInfinite-Dimensional Bayesian Filtering for Detection of Quasiperiodic Phenomena in Spatiotemporal Data.” Physical Review E 88 (5): 052909.
Tait, Daniel J., and Theodoros Damoulas. 2020. β€œVariational Autoencoding of PDE Inverse Problems.” arXiv:2006.15641 [Cs, Stat], June.
Tartakovsky, Alexandre M., Carlos Ortiz Marrero, Paris Perdikaris, Guzel D. Tartakovsky, and David Barajas-Solano. 2018. β€œLearning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks,” August.
Yang, Liu, Dongkun Zhang, and George Em Karniadakis. 2020. β€œPhysics-Informed Generative Adversarial Networks for Stochastic Differential Equations.” SIAM Journal on Scientific Computing 42 (1): A292–317.
Zammit-Mangion, Andrew, Tin Lok James Ng, Quan Vu, and Maurizio Filippone. 2021. β€œDeep Compositional Spatial Models.” Journal of the American Statistical Association 0 (0): 1–22.
Zammit-Mangion, Andrew, and Christopher K. Wikle. 2020. β€œDeep Integro-Difference Equation Models for Spatio-Temporal Forecasting.” Spatial Statistics 37 (June): 100408.
Zang, Yaohua, Gang Bao, Xiaojing Ye, and Haomin Zhou. 2020. β€œWeak Adversarial Networks for High-Dimensional Partial Differential Equations.” Journal of Computational Physics 411 (June): 109409.
Zhang, Dongkun, Ling Guo, and George Em Karniadakis. 2020. β€œLearning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks.” SIAM Journal on Scientific Computing 42 (2): A639–65.
Zhang, Dongkun, Lu Lu, Ling Guo, and George Em Karniadakis. 2019. β€œQuantifying Total Uncertainty in Physics-Informed Neural Networks for Solving Forward and Inverse Stochastic Problems.” Journal of Computational Physics 397 (November): 108850.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.