Neural learning for spatiotemporal systems

2020-09-16 – 2022-07-28
Maybe related 🀷:

The known problems with out-of-sample prediction in neural nets have strong history in spatial inference generally.

On regression from 2d images to 2d images (sometimes 3d is OK) using neural networks. Useful in learning stochastic partial differential equations and other processes, which uses the tools of dynamical NNs and their ilk. Probably handy for machine learning physics and especially PDEs.

In the inverse setting useful tolls might in addition be low rank or lattice Gaussian processes.

I know little about this topic yet. But here are some articles to read.

U-Net

A convnet architecture designed for image segmentation but useful in general image-> image regression (Ronneberger, Fischer, and Brox 2015; Shelhamer, Long, and Darrell 2017).

Fourier operator learning

See ML for PDES.

References

Arridge, Simon, Peter Maass, Ozan Γ–ktem, and Carola-Bibiane SchΓΆnlieb. 2019. β€œSolving Inverse Problems Using Data-Driven Models.” Acta Numerica 28 (May): 1–174.
Ayed, Ibrahim, and Emmanuel de BΓ©zenac. 2019. β€œLearning Dynamical Systems from Partial Observations.” In Advances In Neural Information Processing Systems, 12.
Beck, Christian, Weinan E, and Arnulf Jentzen. 2019. β€œMachine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-Order Backward Stochastic Differential Equations.” Journal of Nonlinear Science 29 (4): 1563–1619.
Dupont, Emilien, Hyunjik Kim, S. M. Ali Eslami, Danilo Jimenez Rezende, and Dan Rosenbaum. 2022. β€œFrom Data to Functa: Your Data Point Is a Function and You Can Treat It Like One.” In Proceedings of the 39th International Conference on Machine Learning, 5694–5725. PMLR.
E, Weinan, Jiequn Han, and Arnulf Jentzen. 2017. β€œDeep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations.” Communications in Mathematics and Statistics 5 (4): 349–80.
E, Weinan, and Bing Yu. 2018. β€œThe Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems.” Communications in Mathematics and Statistics 6 (1): 1–12.
Guth, Stephen Carrol, Alireza Mojahed, and Themistoklis Sapsis. 2023. β€œEvaluation of Machine Learning Architectures on the Quantification Of Epistemic and Aleatoric Uncertainties In Complex Dynamical Systems.” SSRN Scholarly Paper. Rochester, NY.
Han, Jiequn, Arnulf Jentzen, and Weinan E. 2018. β€œSolving High-Dimensional Partial Differential Equations Using Deep Learning.” Proceedings of the National Academy of Sciences 115 (34): 8505–10.
He, QiZhi, David Barajas-Solano, Guzel Tartakovsky, and Alexandre M. Tartakovsky. 2020. β€œPhysics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport.” Advances in Water Resources 141 (July): 103610.
Ikeda, M. 1989. β€œDecentralized Control of Large Scale Systems.” In Three Decades of Mathematical System Theory: A Collection of Surveys at the Occasion of the 50th Birthday of Jan C. Willems, edited by Hendrik Nijmeijer and Johannes M. Schumacher, 219–42. Lecture Notes in Control and Information Sciences. Berlin, Heidelberg: Springer.
Laloy, Eric, and Diederik Jacques. 2019. β€œEmulation of CPU-Demanding Reactive Transport Models: A Comparison of Gaussian Processes, Polynomial Chaos Expansion, and Deep Neural Networks.” Computational Geosciences 23 (5): 1193–1215.
Lu, Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. 2021. β€œDeepXDE: A Deep Learning Library for Solving Differential Equations.” SIAM Review 63 (1): 208–28.
Mo, Shaoxing, Nicholas Zabaras, Xiaoqing Shi, and Jichun Wu. 2019. β€œDeep Autoregressive Neural Networks for High-Dimensional Inverse Problems in Groundwater Contaminant Source Identification.” Water Resources Research 55 (5): 3856–81.
MΓΌller, Johannes, and Marius Zeinhofer. 2020. β€œDeep Ritz Revisited.” arXiv.
Nabian, Mohammad Amin, and Hadi Meidani. 2019. β€œA Deep Learning Solution Approach for High-Dimensional Random Differential Equations.” Probabilistic Engineering Mechanics 57 (July): 14–25.
Park, Ji Hwan, Shinjae Yoo, and Balu Nadiga. 2019. β€œMachine Learning Climate Variability.” In, 5.
Patraucean, Viorica, Ankur Handa, and Roberto Cipolla. 2015. β€œSpatio-Temporal Video Autoencoder with Differentiable Memory.” arXiv:1511.06309 [Cs], November.
Pirinen, Aleksis, Olof Mogren, and MΓ₯rten VΓ€sterdal. 2023. β€œFully Convolutional Networks for Dense Water Flow Intensity Prediction in Swedish Catchment Areas.” arXiv.
Rackauckas, Chris, Alan Edelman, Keno Fischer, Mike Innes, Elliot Saba, Viral B Shah, and Will Tebbutt. 2020. β€œGeneralized Physics-Informed Learning Through Language-Wide Differentiable Programming.” MIT Web Domain, 6.
Raissi, Maziar, P. Perdikaris, and George Em Karniadakis. 2019. β€œPhysics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations.” Journal of Computational Physics 378 (February): 686–707.
Raissi, Maziar, Alireza Yazdani, and George Em Karniadakis. 2020. β€œHidden Fluid Mechanics: Learning Velocity and Pressure Fields from Flow Visualizations.” Science 367 (6481): 1026–30.
Razavi, Saman. 2021. β€œDeep Learning, Explained: Fundamentals, Explainability, and Bridgeability to Process-Based Modelling.” Environmental Modelling & Software 144 (October): 105159.
Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. 2015. β€œU-Net: Convolutional Networks for Biomedical Image Segmentation.” Edited by Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science. Cham: Springer International Publishing.
Ruthotto, Lars, and Eldad Haber. 2020. β€œDeep Neural Networks Motivated by Partial Differential Equations.” Journal of Mathematical Imaging and Vision 62 (3): 352–64.
Safonova, Anastasiia, Gohar Ghazaryan, Stefan Stiller, Magdalena Main-Knorn, Claas Nendel, and Masahiro Ryo. 2023. β€œTen Deep Learning Techniques to Address Small Data Problems with Remote Sensing.” International Journal of Applied Earth Observation and Geoinformation 125 (December): 103569.
SΓ€rkkΓ€, Simo. 2011. β€œLinear Operators and Stochastic Partial Differential Equations in Gaussian Process Regression.” In Artificial Neural Networks and Machine Learning – ICANN 2011, edited by Timo Honkela, WΕ‚odzisΕ‚aw Duch, Mark Girolami, and Samuel Kaski, 6792:151–58. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer.
SΓ€rkkΓ€, Simo, and Jouni Hartikainen. 2012. β€œInfinite-Dimensional Kalman Filtering Approach to Spatio-Temporal Gaussian Process Regression.” In Artificial Intelligence and Statistics.
SΓ€rkkΓ€, Simo, A. Solin, and J. Hartikainen. 2013. β€œSpatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing: A Look at Gaussian Process Regression Through Kalman Filtering.” IEEE Signal Processing Magazine 30 (4): 51–61.
Shankar, Varun, Gavin D Portwood, Arvind T Mohan, Peetak P Mitra, Christopher Rackauckas, Lucas A Wilson, David P Schmidt, and Venkatasubramanian Viswanathan. 2020. β€œLearning Non-Linear Spatio-Temporal Dynamics with Convolutional Neural ODEs.” In Third Workshop on Machine Learning and the Physical Sciences (NeurIPS 2020).
Shelhamer, Evan, Jonathan Long, and Trevor Darrell. 2017. β€œFully Convolutional Networks for Semantic Segmentation.” IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (4): 640–51.
Solin, Arno, and Simo SΓ€rkkΓ€. 2013. β€œInfinite-Dimensional Bayesian Filtering for Detection of Quasiperiodic Phenomena in Spatiotemporal Data.” Physical Review E 88 (5): 052909.
Tait, Daniel J., and Theodoros Damoulas. 2020. β€œVariational Autoencoding of PDE Inverse Problems.” arXiv:2006.15641 [Cs, Stat], June.
Tartakovsky, Alexandre M., Carlos Ortiz Marrero, Paris Perdikaris, Guzel D. Tartakovsky, and David Barajas-Solano. 2018. β€œLearning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks,” August.
Yang, Liu, Dongkun Zhang, and George Em Karniadakis. 2020. β€œPhysics-Informed Generative Adversarial Networks for Stochastic Differential Equations.” SIAM Journal on Scientific Computing 42 (1): A292–317.
Zammit-Mangion, Andrew, Tin Lok James Ng, Quan Vu, and Maurizio Filippone. 2021. β€œDeep Compositional Spatial Models.” Journal of the American Statistical Association 0 (0): 1–22.
Zammit-Mangion, Andrew, and Christopher K. Wikle. 2020. β€œDeep Integro-Difference Equation Models for Spatio-Temporal Forecasting.” Spatial Statistics 37 (June): 100408.
Zang, Yaohua, Gang Bao, Xiaojing Ye, and Haomin Zhou. 2020. β€œWeak Adversarial Networks for High-Dimensional Partial Differential Equations.” Journal of Computational Physics 411 (June): 109409.
Zhang, Dongkun, Ling Guo, and George Em Karniadakis. 2020. β€œLearning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks.” SIAM Journal on Scientific Computing 42 (2): A639–65.
Zhang, Dongkun, Lu Lu, Ling Guo, and George Em Karniadakis. 2019. β€œQuantifying Total Uncertainty in Physics-Informed Neural Networks for Solving Forward and Inverse Stochastic Problems.” Journal of Computational Physics 397 (November): 108850.

Download bibliography

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.