Inverse problems where the model parameter is in some function space. For me this usually implies a spatiotemporal model, usually in the context of PDE solvers, particularly approximate ones.

Inverse problems arise naturally in tomography, compressed sensing, deconvolution, inverting PDEs and many other areas.

Suppose I have a PDE, possibly with some unknown parameters in the driving equation.
All being equal I can do not too badly at approximating that with tools already mentioned.
What if I wish to simultaneously infer some unknown *inputs*?
Then we consider it as an inverse problem.
This is not quite the same as the predictive problem that many of the methods consider.
However, we are free to use simulation-based inference to solve, or MCMC methods to do so for any of the forward-operator-learning approaches.
To train the model to solve the inverse problem directly, we might consider GANs or variational inference.
At this point we are more or less required to start using a probabilistic network or we will miss essential uncertainty quantification.

We might also be inclined to use approximate methods. We are surprised to find anything like a clean closed-form solution for the posterior distribution of some parameter in a PDE. Why would it?

As for how we might proceed, Liu, Yeo, and Lu (2020) is one approach, generalizing the approach of F. Sigrist, KΓΌnsch, and Stahel (2015b), but for advection/diffusion equations specifically. Generic methods include Bao et al. (2020); Jo et al. (2020); Lu, Mao, and Meng (2019); Raissi, Perdikaris, and Karniadakis (2019); Tait and Damoulas (2020); Xu and Darve (2020); Yang, Zhang, and Karniadakis (2020); Zhang, Guo, and Karniadakis (2020); Zhang et al. (2019).

## Bayesian nonparametrics

Since this kind of problem naturally invites functional parameters, we are in the world of Bayesian nonparametrics, which has a slightly different notation than you usually see in Bayes textbooks. I suspect that there is a useful role for various Bayesian nonparametrics here, but the easiest of all is Gaussian process, which I handle next:

## Gaussian process parameters

Alexanderian (2021) states a βwell-knownβ result, that the solution of a Bayesian linear inverse problem with Gaussian prior and noise models is a Gaussian posterior \(\mu_{\text {post }}^{y}=\mathcal{N}\left(m_{\text {MAP }}, \mathcal{C}_{\text {post }}\right)\), where \[ \mathcal{C}_{\text {post }}=\left(\mathcal{F}^{*} \boldsymbol{\Gamma}_{\text {noise }}^{-1} \mathcal{F}+\mathcal{C}_{\text {pr }}^{-1}\right)^{-1} \quad \text { and } \quad m_{\text {MAP }}=\mathcal{C}_{\text {post }}\left(\mathcal{F}^{*} \boldsymbol{\Gamma}_{\text {noise }}^{-1} \boldsymbol{y}+\mathcal{C}_{\text {pr }}^{-1} m_{\text {MAP }}\right). \]

## Finite Element Models

Finite Element Models of PDEs of PDEs (annd possibly other representation?) of PDES can be expressed as locally-linear constraints and thus expressed using Gaussian Belief Propagation (Y. El-Kurdi et al. 2016; Y. M. El-Kurdi 2014; Y. El-Kurdi et al. 2015). Note that in this setting, there is nothing special about the inversion process. Inference proceeds the same either way, as a variational message passing algorithm.

## References

*arXiv:2005.12998 [Math]*, January.

*Inverse Problems*36 (11): 115003.

*Proceedings of the National Academy of Sciences*117 (10): 5242β49.

*SIAM Journal on Scientific Computing*35 (6): A2494β2523.

*Inverse Problems & Imaging*10 (4): 943.

*SIAM Journal on Numerical Analysis*48 (1): 322β45.

*The Annals of Statistics*21 (2): 903β23.

*Proceedings of the National Academy of Sciences*, May.

*arXiv:1302.6989 [Math]*, July.

*Computer Physics Communications*193 (August): 38β48.

*IEEE Transactions on Magnetics*52 (3): 1β4.

*Econometric Theory*32 (1): 71β121.

*Journal of Mathematical Analysis and Applications*31 (3): 682β716.

*arXiv:1610.08035 [Stat]*.

*Networks & Heterogeneous Media*15 (2): 247.

*Statistical and Computational Inverse Problems*. Applied Mathematical Sciences. New York: Springer-Verlag.

*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*63 (3): 425β64.

*The Annals of Statistics*39 (5).

*Inverse Problems & Imaging*6 (2): 267.

*Inverse Problems and Imaging*3 (1): 87β122.

*Journal of the American Statistical Association*0 (0): 1β18.

*arXiv:1910.03193 [Cs, Stat]*, April.

*SIAM Review*63 (1): 208β28.

*arXiv:2004.12550 [Stat]*, October.

*International Geophysics*, 81:237β65. Elsevier.

*Reliability Engineering & System Safety*, The Fourth International Conference on Sensitivity Analysis of Model Output (SAMO 2004), 91 (10): 1290β300.

*Journal of the Royal Society, Interface*13 (118): 20151107.

*SIAM Journal on Scientific Computing*36 (4): A1525β55.

*Journal of the American Statistical Association*112 (519): 1274β85.

*Journal of Computational Physics*378 (February): 686β707.

*Inverse Problems & Imaging*8 (2): 561.

*IEEE Signal Processing Magazine*30 (4): 51β61.

*Journal of Statistical Software*63 (14).

*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*77 (1): 3β33.

*Acta Numerica*19: 451β559.

*arXiv:1603.02004 [Math]*, December.

*arXiv:2006.15641 [Cs, Stat]*, June.

*Inverse Problem Theory and Methods for Model Parameter Estimation*. SIAM.

*Mapping Of Probabilities*.

*arXiv:1909.00232 [Cs, Math, Stat]*, July.

*pyEMU: A Python Framework for Environmental Model Uncertainty Analysis Version .01*. U.S. Geological Survey.

*Environmental Modelling & Software*85 (November): 217β28.

*arXiv:2011.11955 [Cs, Math]*.

*SIAM Journal on Scientific Computing*42 (1): A292β317.

*Geoscientific Model Development Discussions*, July, 1β51.

*SIAM Journal on Scientific Computing*42 (2): A639β65.

*Journal of Computational Physics*397 (November): 108850.

## No comments yet. Why not leave one?