The known problems with out-of-sample prediction in neural nets have strong history in spatial inference generally.
On regression from 2d images to 2d images (sometimes 3d is OK) using neural networks. Useful in learning stochastic partial differential equations and other processes, which uses the tools of dynamical NNs and their ilk. Probably handy for machine learning physics and especially PDEs.
In the inverse setting useful tolls might in addition be low rank or lattice Gaussian processes.
I know little about this topic yet. But here are some articles to read.
U-Net
A convnet architecture designed for image segmentation but useful in general image-> image regression (Ronneberger, Fischer, and Brox 2015; Shelhamer, Long, and Darrell 2017).
Fourier operator learning
See ML for PDES.
References
Arridge, Simon, Peter Maass, Ozan Öktem, and Carola-Bibiane Schönlieb. 2019. “Solving Inverse Problems Using Data-Driven Models.” Acta Numerica 28 (May): 1–174.
Ayed, Ibrahim, and Emmanuel de Bézenac. 2019. “Learning Dynamical Systems from Partial Observations.” In Advances In Neural Information Processing Systems, 12.
Beck, Christian, Weinan E, and Arnulf Jentzen. 2019. “Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-Order Backward Stochastic Differential Equations.” Journal of Nonlinear Science 29 (4): 1563–1619.
Dupont, Emilien, Hyunjik Kim, S. M. Ali Eslami, Danilo Rezende, and Dan Rosenbaum. 2022. “From Data to Functa: Your Data Point Is a Function and You Can Treat It Like One.” arXiv.
E, Weinan, Jiequn Han, and Arnulf Jentzen. 2017. “Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations.” Communications in Mathematics and Statistics 5 (4): 349–80.
E, Weinan, and Bing Yu. 2018. “The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems.” Communications in Mathematics and Statistics 6 (1): 1–12.
Han, Jiequn, Arnulf Jentzen, and Weinan E. 2018. “Solving High-Dimensional Partial Differential Equations Using Deep Learning.” Proceedings of the National Academy of Sciences 115 (34): 8505–10.
He, QiZhi, David Barajas-Solano, Guzel Tartakovsky, and Alexandre M. Tartakovsky. 2020. “Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport.” Advances in Water Resources 141 (July): 103610.
Ikeda, M. 1989. “Decentralized Control of Large Scale Systems.” In Three Decades of Mathematical System Theory: A Collection of Surveys at the Occasion of the 50th Birthday of Jan C. Willems, edited by Hendrik Nijmeijer and Johannes M. Schumacher, 219–42. Lecture Notes in Control and Information Sciences. Berlin, Heidelberg: Springer.
Laloy, Eric, and Diederik Jacques. 2019. “Emulation of CPU-Demanding Reactive Transport Models: A Comparison of Gaussian Processes, Polynomial Chaos Expansion, and Deep Neural Networks.” Computational Geosciences 23 (5): 1193–1215.
Lu, Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. 2021. “DeepXDE: A Deep Learning Library for Solving Differential Equations.” SIAM Review 63 (1): 208–28.
Mo, Shaoxing, Nicholas Zabaras, Xiaoqing Shi, and Jichun Wu. 2019. “Deep Autoregressive Neural Networks for High-Dimensional Inverse Problems in Groundwater Contaminant Source Identification.” Water Resources Research 55 (5): 3856–81.
Müller, Johannes, and Marius Zeinhofer. 2020. “Deep Ritz Revisited.” arXiv.
Nabian, Mohammad Amin, and Hadi Meidani. 2019. “A Deep Learning Solution Approach for High-Dimensional Random Differential Equations.” Probabilistic Engineering Mechanics 57 (July): 14–25.
Park, Ji Hwan, Shinjae Yoo, and Balu Nadiga. 2019. “Machine Learning Climate Variability.” In, 5.
Patraucean, Viorica, Ankur Handa, and Roberto Cipolla. 2015. “Spatio-Temporal Video Autoencoder with Differentiable Memory.” arXiv:1511.06309 [Cs], November.
Pirinen, Aleksis, Olof Mogren, and Mårten Västerdal. 2023. “Fully Convolutional Networks for Dense Water Flow Intensity Prediction in Swedish Catchment Areas.” arXiv.
Rackauckas, Chris, Alan Edelman, Keno Fischer, Mike Innes, Elliot Saba, Viral B Shah, and Will Tebbutt. 2020. “Generalized Physics-Informed Learning Through Language-Wide Differentiable Programming.” MIT Web Domain, 6.
Raissi, Maziar, P. Perdikaris, and George Em Karniadakis. 2019. “Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations.” Journal of Computational Physics 378 (February): 686–707.
Raissi, Maziar, Alireza Yazdani, and George Em Karniadakis. 2020. “Hidden Fluid Mechanics: Learning Velocity and Pressure Fields from Flow Visualizations.” Science 367 (6481): 1026–30.
Razavi, Saman. 2021. “Deep Learning, Explained: Fundamentals, Explainability, and Bridgeability to Process-Based Modelling.” Environmental Modelling & Software 144 (October): 105159.
Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. 2015. “U-Net: Convolutional Networks for Biomedical Image Segmentation.” Edited by Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science. Cham: Springer International Publishing.
Ruthotto, Lars, and Eldad Haber. 2020. “Deep Neural Networks Motivated by Partial Differential Equations.” Journal of Mathematical Imaging and Vision 62 (3): 352–64.
Särkkä, Simo. 2011. “Linear Operators and Stochastic Partial Differential Equations in Gaussian Process Regression.” In Artificial Neural Networks and Machine Learning – ICANN 2011, edited by Timo Honkela, Włodzisław Duch, Mark Girolami, and Samuel Kaski, 6792:151–58. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer.
Särkkä, Simo, and Jouni Hartikainen. 2012. “Infinite-Dimensional Kalman Filtering Approach to Spatio-Temporal Gaussian Process Regression.” In Artificial Intelligence and Statistics.
Särkkä, Simo, A. Solin, and J. Hartikainen. 2013. “Spatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing: A Look at Gaussian Process Regression Through Kalman Filtering.” IEEE Signal Processing Magazine 30 (4): 51–61.
Shankar, Varun, Gavin D Portwood, Arvind T Mohan, Peetak P Mitra, Christopher Rackauckas, Lucas A Wilson, David P Schmidt, and Venkatasubramanian Viswanathan. 2020. “Learning Non-Linear Spatio-Temporal Dynamics with Convolutional Neural ODEs.” In Third Workshop on Machine Learning and the Physical Sciences (NeurIPS 2020).
Shelhamer, Evan, Jonathan Long, and Trevor Darrell. 2017. “Fully Convolutional Networks for Semantic Segmentation.” IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (4): 640–51.
Solin, Arno, and Simo Särkkä. 2013. “Infinite-Dimensional Bayesian Filtering for Detection of Quasiperiodic Phenomena in Spatiotemporal Data.” Physical Review E 88 (5): 052909.
Tait, Daniel J., and Theodoros Damoulas. 2020. “Variational Autoencoding of PDE Inverse Problems.” arXiv:2006.15641 [Cs, Stat], June.
Tartakovsky, Alexandre M., Carlos Ortiz Marrero, Paris Perdikaris, Guzel D. Tartakovsky, and David Barajas-Solano. 2018. “Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks,” August.
Yang, Liu, Dongkun Zhang, and George Em Karniadakis. 2020. “Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations.” SIAM Journal on Scientific Computing 42 (1): A292–317.
Zammit-Mangion, Andrew, Tin Lok James Ng, Quan Vu, and Maurizio Filippone. 2021. “Deep Compositional Spatial Models.” Journal of the American Statistical Association 0 (0): 1–22.
Zammit-Mangion, Andrew, and Christopher K. Wikle. 2020. “Deep Integro-Difference Equation Models for Spatio-Temporal Forecasting.” Spatial Statistics 37 (June): 100408.
Zang, Yaohua, Gang Bao, Xiaojing Ye, and Haomin Zhou. 2020. “Weak Adversarial Networks for High-Dimensional Partial Differential Equations.” Journal of Computational Physics 411 (June): 109409.
Zhang, Dongkun, Ling Guo, and George Em Karniadakis. 2020. “Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks.” SIAM Journal on Scientific Computing 42 (2): A639–65.
Zhang, Dongkun, Lu Lu, Ling Guo, and George Em Karniadakis. 2019. “Quantifying Total Uncertainty in Physics-Informed Neural Networks for Solving Forward and Inverse Stochastic Problems.” Journal of Computational Physics 397 (November): 108850.
No comments yet. Why not leave one?