Inverse problems for PDEs and other spatial models

a.k.a. Bayesian calibration, model uncertainty

Inverse problems where the model is full of spatioemtporal correlations. I am particularly thinking about this in the context of PDE solvers, particularly approximate ones.

Suppose I have a PDE, possibly with some unknown parameters in the driving equation. All being equal I can do not too badly at approximating that with tools already mentioned. What if I wish simultaneously infer some unknown inputs? Then we consider it as an inverse problem. This is not quite the same as the predictive problem that many of the methods consider. However, we are free to use simulation-based inference to solve, or MCMC methods to do so for any of the forward-operator-learning approaches. To train the model to solve target the inverse problem directly, we might consider GANs or variational inference. At this point we are more or less required to start thinking about this in a probabilistic network or we will miss essential uncertainty quantification.

We are also encouraged to think about this as an approximation problem. We are surprised to find anything like a clean closed-form solution for the posterior distribution of some parameter in a PDE. Worse, the hypothetical solution is probably not even particularly computationally convenient. Why would it be? This will be especially trick in Bayesian inversion.

As far as how we might proceed, Liu, Yeo, and Lu (2020) is one approach, generalizing the approach of F. Sigrist, KΓΌnsch, and Stahel (2015b), but for advection/diffusion equations specifically. Generic methods include Bao et al. (2020); Jo et al. (2019); Lu, Mao, and Meng (2019); Raissi, Perdikaris, and Karniadakis (2019); Tait and Damoulas (2020); Xu and Darve (2020); Yang, Zhang, and Karniadakis (2020); Zhang, Guo, and Karniadakis (2020); Zhang et al. (2019).

Gaussian process case

Alexanderian (2021) states a β€˜well-known’ result, that the solution of a Bayesian linear inverse problem with Gaussian prior and noise models is a Gaussian posterior measure \(\mu_{\text {post }}^{y}=\mathcal{N}\left(m_{\text {MAP }}, \mathcal{C}_{\text {post }}\right)\), where \[ \mathcal{C}_{\text {post }}=\left(\mathcal{F}^{*} \boldsymbol{\Gamma}_{\text {noise }}^{-1} \mathcal{F}+\mathcal{C}_{\text {pr }}^{-1}\right)^{-1} \quad \text { and } \quad m_{\text {MAP }}=\mathcal{C}_{\text {post }}\left(\mathcal{F}^{*} \boldsymbol{\Gamma}_{\text {noise }}^{-1} \boldsymbol{y}+\mathcal{C}_{\text {pr }}^{-1} m_{\text {MAP }}\right) \]


Alexanderian, Alen. 2021. β€œOptimal Experimental Design for Infinite-Dimensional Bayesian Inverse Problems Governed by PDEs: A Review.” arXiv:2005.12998 [Math], January.
Bao, Gang, Xiaojing Ye, Yaohua Zang, and Haomin Zhou. 2020. β€œNumerical Solution of Inverse Problems by Weak Adversarial Networks.” Inverse Problems 36 (11): 115003.
Brehmer, Johann, Gilles Louppe, Juan Pavez, and Kyle Cranmer. 2020. β€œMining Gold from Implicit Models to Improve Likelihood-Free Inference.” Proceedings of the National Academy of Sciences 117 (10): 5242–49.
Cotter, S. L., M. Dashti, and A. M. Stuart. 2010. β€œApproximation of Bayesian Inverse Problems for PDEs.” SIAM Journal on Numerical Analysis 48 (1): 322–45.
Cranmer, Kyle, Johann Brehmer, and Gilles Louppe. 2020. β€œThe Frontier of Simulation-Based Inference.” Proceedings of the National Academy of Sciences, May.
Dashti, Masoumeh, and Andrew M. Stuart. 2015. β€œThe Bayesian Approach To Inverse Problems.” arXiv:1302.6989 [Math], July.
Franklin, Joel N. 1970. β€œWell-Posed Stochastic Extensions of Ill-Posed Linear Problems.” Journal of Mathematical Analysis and Applications 31 (3): 682–716.
Grigorievskiy, Alexander, Neil Lawrence, and Simo SΓ€rkkΓ€. 2017. β€œParallelizable Sparse Inverse Formulation Gaussian Processes (SpInGP).” In arXiv:1610.08035 [Stat].
Jo, Hyeontae, Hwijae Son, Hyung Ju Hwang, and Eunheui Kim. 2019. β€œDeep Neural Network Approach to Forward-Inverse Problems.” arXiv:1907.12925 [Cs, Math], July.
Kaipio, Jari, and E. Somersalo. 2005. Statistical and Computational Inverse Problems. Applied Mathematical Sciences. New York: Springer-Verlag.
Kennedy, Marc C., and Anthony O’Hagan. 2001. β€œBayesian Calibration of Computer Models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63 (3): 425–64.
Lasanen, Sari. 2012. β€œNon-Gaussian Statistical Inverse Problems. Part II: Posterior Convergence for Approximated Unknowns.” Inverse Problems & Imaging 6 (2): 267.
Lassas, Matti, Eero Saksman, and Samuli Siltanen. 2009. β€œDiscretization-Invariant Bayesian Inversion and Besov Space Priors.” Inverse Problems and Imaging 3 (1): 87–122.
Liu, Xiao, Kyongmin Yeo, and Siyuan Lu. 2020. β€œStatistical Modeling for Spatio-Temporal Data From Stochastic Convection-Diffusion Processes.” Journal of the American Statistical Association 0 (0): 1–18.
Lu, Lu, Pengzhan Jin, and George Em Karniadakis. 2020. β€œDeepONet: Learning Nonlinear Operators for Identifying Differential Equations Based on the Universal Approximation Theorem of Operators.” arXiv:1910.03193 [Cs, Stat], April.
Lu, Lu, Zhiping Mao, and Xuhui Meng. 2019. β€œDeepXDE: A Deep Learning Library for Solving Differential Equations.” In, 6.
Lu, Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. 2021. β€œDeepXDE: A Deep Learning Library for Solving Differential Equations.” SIAM Review 63 (1): 208–28.
Mosegaard, Klaus, and Albert Tarantola. 2002. β€œProbabilistic Approach to Inverse Problems.” In International Geophysics, 81:237–65. Elsevier.
O’Hagan, A. 2006. β€œBayesian Analysis of Computer Code Outputs: A Tutorial.” Reliability Engineering & System Safety, The Fourth International Conference on Sensitivity Analysis of Model Output (SAMO 2004), 91 (10): 1290–300.
Perdikaris, Paris, and George Em Karniadakis. 2016. β€œModel inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond.” Journal of the Royal Society, Interface 13 (118): 20151107.
Plumlee, Matthew. 2017. β€œBayesian Calibration of Inexact Computer Models.” Journal of the American Statistical Association 112 (519): 1274–85.
Raissi, Maziar, Paris Perdikaris, and George Em Karniadakis. 2017a. β€œPhysics Informed Deep Learning (Part I): Data-Driven Solutions of Nonlinear Partial Differential Equations,” November.
β€”β€”β€”. 2017b. β€œPhysics Informed Deep Learning (Part II): Data-Driven Discovery of Nonlinear Partial Differential Equations,” November.
Raissi, Maziar, P. Perdikaris, and George Em Karniadakis. 2019. β€œPhysics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations.” Journal of Computational Physics 378 (February): 686–707.
Roininen, Lassi, Janne M. J. Huttunen, and Sari Lasanen. 2014. β€œWhittle-MatΓ©rn Priors for Bayesian Statistical Inversion with Applications in Electrical Impedance Tomography.” Inverse Problems & Imaging 8 (2): 561.
SΓ€rkkΓ€, Simo, A. Solin, and J. Hartikainen. 2013. β€œSpatiotemporal Learning via Infinite-Dimensional Bayesian Filtering and Smoothing: A Look at Gaussian Process Regression Through Kalman Filtering.” IEEE Signal Processing Magazine 30 (4): 51–61.
Sigrist, Fabio Roman Albert. 2013. β€œPhysics Based Dynamic Modeling of Space-Time Data.” Application/pdf. ETH Zurich.
Sigrist, Fabio, Hans R. KΓΌnsch, and Werner A. Stahel. 2015a. β€œSpate : An R Package for Spatio-Temporal Modeling with a Stochastic Advection-Diffusion Process.” Application/pdf. Journal of Statistical Software 63 (14).
β€”β€”β€”. 2015b. β€œStochastic Partial Differential Equation Based Modelling of Large Space-Time Data Sets.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 77 (1): 3–33.
Stuart, A. M. 2010. β€œInverse Problems: A Bayesian Perspective.” Acta Numerica 19: 451–559.
Stuart, Andrew M., and Aretha L. Teckentrup. 2016. β€œPosterior Consistency for Gaussian Process Approximations of Bayesian Posterior Distributions.” arXiv:1603.02004 [Math], December.
Tait, Daniel J., and Theodoros Damoulas. 2020. β€œVariational Autoencoding of PDE Inverse Problems.” arXiv:2006.15641 [Cs, Stat], June.
Tarantola, Albert. 2005. Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM.
β€”β€”β€”. n.d. Mapping Of Probabilities.
Teckentrup, Aretha L. 2020. β€œConvergence of Gaussian Process Regression with Estimated Hyper-Parameters and Applications in Bayesian Inverse Problems.” arXiv:1909.00232 [Cs, Math, Stat], July.
Welter, David E., Jeremy T. White, Randall J. Hunt, and John E. Doherty. 2015. β€œApproaches in Highly Parameterized Inversionβ€”PEST++ Version 3, a Parameter ESTimation and Uncertainty Analysis Software Suite Optimized for Large Environmental Models.” USGS Numbered Series 7-C12. Techniques and Methods. Reston, VA: U.S. Geological Survey.
White, Jeremy T., Michael N. Fienen, and John E. Doherty. 2016a. pyEMU: A Python Framework for Environmental Model Uncertainty Analysis Version .01. U.S. Geological Survey.
β€”β€”β€”. 2016b. β€œA Python Framework for Environmental Model Uncertainty Analysis.” Environmental Modelling & Software 85 (November): 217–28.
Xu, Kailai, and Eric Darve. 2020. β€œADCME: Learning Spatially-Varying Physical Fields Using Deep Neural Networks.” In arXiv:2011.11955 [Cs, Math].
Yang, Liu, Dongkun Zhang, and George Em Karniadakis. 2020. β€œPhysics-Informed Generative Adversarial Networks for Stochastic Differential Equations.” SIAM Journal on Scientific Computing 42 (1): A292–317.
Zammit-Mangion, Andrew, Michael Bertolacci, Jenny Fisher, Ann Stavert, Matthew L. Rigby, Yi Cao, and Noel Cressie. 2021. β€œWOMBAT v1.0: A fully Bayesian global flux-inversion framework.” Geoscientific Model Development Discussions, July, 1–51.
Zhang, Dongkun, Ling Guo, and George Em Karniadakis. 2020. β€œLearning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks.” SIAM Journal on Scientific Computing 42 (2): A639–65.
Zhang, Dongkun, Lu Lu, Ling Guo, and George Em Karniadakis. 2019. β€œQuantifying Total Uncertainty in Physics-Informed Neural Networks for Solving Forward and Inverse Stochastic Problems.” Journal of Computational Physics 397 (November): 108850.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.