# Inverse problems

March 30, 2016 — June 30, 2022

functional analysis
linear algebra
probability
sparser than thou
statistics

Robert Ackroyd introduces some handy phrasing for the connections between statistical estimation theory and inverse problem solving.

Photogrammetry, MRIs, SLAM, volumetric reconstruction and X-ray crystallography are all examples of inverse problems. any of these can be constructed as classical belief propagaation, especially Gaussian BP, or in a basic case least squares.

I happen to think that this is a case where it is much easier to explain in terms of Bayesian inference, so my attempt at an actual explanation is under Bayesian inverse problems.

We can do it in terms of frequentist methods, but it does not add much in the way of explanatory value; we end up considering regularizers instead of priors, but the working in between is pretty much the same. (IMO) However, in doing so we focus on point estimates rather than entire densities, which encourages us to solve the problem by optimisation rather than integration, which is a useful insight, for example, when we consider Laplace approximations.

## 1 Domain-specific model inversion

PEST, PEST++, and pyemu are some integrated systems for uncertainty quantification that use some weird terminology, such a FOSM (First-order-second-moment) models. They use various linear-algebra tricks to find plausible subspaces and samples.

See also xinvert :

This is a Python package to invert or solve many classical problems in atmospheric sciences and physical oceanography. This geophysical fluid dynamics (GFD) problems are formulated as second-order partial differential equations (PDEs), and can be inverted using success-over relaxation (SOR) iteration with proper boundary conditions. This project is published on GitHub and can be cited using its Zenodo DOI.

## 2 Interesting specific techniques

Leaning to reconstruct introduces partly-learned, partly designed reconstruction operator trick. 🏗️

## 3 Radiance fields

A fun way of reconstructing objects from photos; differentiable photogrammetry.

## 4 References

Adler, and Öktem. 2018. IEEE Transactions on Medical Imaging.
Alberti, De Vito, Lassas, et al. 2021. arXiv:2106.06513 [Cs, Math, Stat].
Aster, Borchers, and Thurber. 2019. Parameter Estimation and Inverse Problems.
Basir, and Senocak. 2022. Journal of Computational Physics.
Bissantz, Hohage, and Munk. 2004. Inverse Problems.
Borcea, Druskin, and Knizhnerman. 2005. Communications on Pure and Applied Mathematics.
Borgerding, and Schniter. 2016. arXiv:1612.01183 [Cs, Math].
Brehmer, Louppe, Pavez, et al. 2020. Proceedings of the National Academy of Sciences.
Bui-Thanh. 2012.
Chen, and Oliver. 2013. Computational Geosciences.
Cranmer, Brehmer, and Louppe. 2020. Proceedings of the National Academy of Sciences.
Daubechies, Defrise, and De Mol. 2004. Communications on Pure and Applied Mathematics.
Engl, Hofinger, and Kindermann. 2005. Inverse Problems.
Engl, and Nashed. 1981. Journal of Mathematical Analysis and Applications.
Fernández-Martínez, Fernández-Muñiz, Pallero, et al. 2013. Journal of Applied Geophysics.
Grigorievskiy, Lawrence, and Särkkä. 2017. In arXiv:1610.08035 [Stat].
Holl, Koltun, and Thuerey. 2022. In.
Kaipio, and Somersalo. 2005. Statistical and Computational Inverse Problems. Applied Mathematical Sciences.
Kaipio, and Somersalo. 2007. Journal of Computational and Applied Mathematics.
Lehtinen, Paivarinta, and Somersalo. 1989. Inverse Problems.
Mandelbaum. 1984. Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete.
Mosegaard, and Tarantola. 1995. Journal of Geophysical Research: Solid Earth.
———. 2002. In International Geophysics. International Handbook of Earthquake and Engineering Seismology, Part A.
Murray-Smith, and Pearlmutter. 2005. In Deterministic and Statistical Methods in Machine Learning. Lecture Notes in Computer Science.
O’Callaghan, and Ramos. 2011. In Twenty-Fifth AAAI Conference on Artificial Intelligence.
O’Sullivan. 1986. Statistical Science.
Oliver. 2022. Mathematical Geosciences.
Pikkarainen. 2006. Inverse Problems.
Plumlee. 2017. Journal of the American Statistical Association.
Putzky, and Welling. 2017. arXiv:1706.04008 [Cs].
Qian. 2023. Journal of Open Source Software.
Schnell, Holl, and Thuerey. 2022. arXiv:2203.10131 [Physics].
Schwab, and Stuart. 2012. Inverse Problems.
Stuart. 2010. Acta Numerica.
Sun, Scanlon, Save, et al. 2021. Water Resources Research.
Tait, and Damoulas. 2020. arXiv:2006.15641 [Cs, Stat].
Tarantola. 2005. Inverse Problem Theory and Methods for Model Parameter Estimation.
Tonolini, Radford, Turpin, et al. 2020. Journal of Machine Learning Research.
Tropp, and Wright. 2010. Proceedings of the IEEE.
Wei, Fan, Carin, et al. 2017. arXiv:1709.01841 [Cs].
Welter, Doherty, Hunt, et al. 2012.
Welter, White, Hunt, et al. 2015. USGS Numbered Series 7-C12. Techniques and Methods.
White. 2018. Environmental Modelling & Software.
White, Fienen, Barlow, et al. 2018. Environmental Modelling & Software.
White, Fienen, and Doherty. 2016a.
———. 2016b. Environmental Modelling & Software.
White, Hunt, Fienen, et al. 2020. USGS Numbered Series 7-C26. Techniques and Methods.
Zammit-Mangion, Bertolacci, Fisher, et al. 2021. Geoscientific Model Development Discussions.
Zhang, Lu, Guo, et al. 2019. Journal of Computational Physics.